Jouons avec JSON

 

Le très gros point fort du JSON est qu’il est très peu verbeux (contrairement au XML) et performant à analyser (contrairement au …). C’est aussi un point faible pour nous administrateurs, humains, qui avons du mal à le comprendre. Pour cela, il faut pouvoir le lire. Ce n’est pas toujours évident, comme le montre cet exemple :

{"_shards":{"total":90,"successful":45,"failed":0},"_all":{"primaries":{"docs":{"count":99254,"deleted":0},"store":{"size_in_bytes":192367723,"throttle_time_in_millis":28623},"indexing":{"index_total":99254,"index_time_in_millis":218667,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":24550,"time_in_millis":2000,"exists_total":24534,"exists_time_in_millis":2000,"missing_total":16,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":993640,"query_time_in_millis":4358463,"query_current":0,"fetch_total":197801,"fetch_time_in_millis":40594,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":773,"total_time_in_millis":43183},"flush":{"total":50,"total_time_in_millis":12412},"warmer":{"current":0,"total":822,"total_time_in_millis":1275},"filter_cache":{"memory_size_in_bytes":3495180,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":32400201,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":238,"memory_in_bytes":1600228},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":99254,"deleted":0},"store":{"size_in_bytes":192367723,"throttle_time_in_millis":28623},"indexing":{"index_total":99254,"index_time_in_millis":218667,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":24550,"time_in_millis":2000,"exists_total":24534,"exists_time_in_millis":2000,"missing_total":16,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":993640,"query_time_in_millis":4358463,"query_current":0,"fetch_total":197801,"fetch_time_in_millis":40594,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":773,"total_time_in_millis":43183},"flush":{"total":50,"total_time_in_millis":12412},"warmer":{"current":0,"total":822,"total_time_in_millis":1275},"filter_cache":{"memory_size_in_bytes":3495180,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":32400201,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":238,"memory_in_bytes":1600228},"translog":{"operations":0,"size_in_bytes":0}}},"indices":{"vm-dev_123_2014-10-16_19-04-medias":{"primaries":{"docs":{"count":43053,"deleted":0},"store":{"size_in_bytes":9326582,"throttle_time_in_millis":3059},"indexing":{"index_total":43053,"index_time_in_millis":13411,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":0,"query_time_in_millis":0,"query_current":0,"fetch_total":0,"fetch_time_in_millis":0,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":199,"total_time_in_millis":2249},"flush":{"total":10,"total_time_in_millis":5592},"warmer":{"current":0,"total":220,"total_time_in_millis":119},"filter_cache":{"memory_size_in_bytes":0,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":0,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":15,"memory_in_bytes":338520},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":43053,"deleted":0},"store":{"size_in_bytes":9326582,"throttle_time_in_millis":3059},"indexing":{"index_total":43053,"index_time_in_millis":13411,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":0,"query_time_in_millis":0,"query_current":0,"fetch_total":0,"fetch_time_in_millis":0,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":199,"total_time_in_millis":2249},"flush":{"total":10,"total_time_in_millis":5592},"warmer":{"current":0,"total":220,"total_time_in_millis":119},"filter_cache":{"memory_size_in_bytes":0,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":0,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":15,"memory_in_bytes":338520},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_16_2014-10-21_12-17-artworks":{"primaries":{"docs":{"count":2348,"deleted":0},"store":{"size_in_bytes":9316186,"throttle_time_in_millis":0},"indexing":{"index_total":2348,"index_time_in_millis":7756,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":4702,"time_in_millis":678,"exists_total":4686,"exists_time_in_millis":678,"missing_total":16,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":407110,"query_time_in_millis":429377,"query_current":0,"fetch_total":106584,"fetch_time_in_millis":13256,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":32,"total_time_in_millis":731},"flush":{"total":5,"total_time_in_millis":219},"warmer":{"current":0,"total":37,"total_time_in_millis":13},"filter_cache":{"memory_size_in_bytes":99792,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":4375465,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":27,"memory_in_bytes":63977},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":2348,"deleted":0},"store":{"size_in_bytes":9316186,"throttle_time_in_millis":0},"indexing":{"index_total":2348,"index_time_in_millis":7756,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":4702,"time_in_millis":678,"exists_total":4686,"exists_time_in_millis":678,"missing_total":16,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":407110,"query_time_in_millis":429377,"query_current":0,"fetch_total":106584,"fetch_time_in_millis":13256,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":32,"total_time_in_millis":731},"flush":{"total":5,"total_time_in_millis":219},"warmer":{"current":0,"total":37,"total_time_in_millis":13},"filter_cache":{"memory_size_in_bytes":99792,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":4375465,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":27,"memory_in_bytes":63977},"translog":{"operations":0,"size_in_bytes":0}}},"vm-stg_29_2014-10-23_17-34-artworks":{"primaries":{"docs":{"count":1839,"deleted":0},"store":{"size_in_bytes":6950220,"throttle_time_in_millis":0},"indexing":{"index_total":1839,"index_time_in_millis":26329,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":857,"time_in_millis":36,"exists_total":857,"exists_time_in_millis":36,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":54700,"query_time_in_millis":48757,"query_current":0,"fetch_total":3876,"fetch_time_in_millis":395,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":28,"total_time_in_millis":7022},"flush":{"total":5,"total_time_in_millis":374},"warmer":{"current":0,"total":33,"total_time_in_millis":73},"filter_cache":{"memory_size_in_bytes":100968,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":4120891,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":23,"memory_in_bytes":53502},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":1839,"deleted":0},"store":{"size_in_bytes":6950220,"throttle_time_in_millis":0},"indexing":{"index_total":1839,"index_time_in_millis":26329,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":857,"time_in_millis":36,"exists_total":857,"exists_time_in_millis":36,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":54700,"query_time_in_millis":48757,"query_current":0,"fetch_total":3876,"fetch_time_in_millis":395,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":28,"total_time_in_millis":7022},"flush":{"total":5,"total_time_in_millis":374},"warmer":{"current":0,"total":33,"total_time_in_millis":73},"filter_cache":{"memory_size_in_bytes":100968,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":4120891,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":23,"memory_in_bytes":53502},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_28_2014-10-18_19-43-artworks":{"primaries":{"docs":{"count":6781,"deleted":0},"store":{"size_in_bytes":16296415,"throttle_time_in_millis":1465},"indexing":{"index_total":6781,"index_time_in_millis":18053,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":79900,"query_time_in_millis":210164,"query_current":0,"fetch_total":8701,"fetch_time_in_millis":1953,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":63,"total_time_in_millis":3943},"flush":{"total":5,"total_time_in_millis":557},"warmer":{"current":0,"total":68,"total_time_in_millis":198},"filter_cache":{"memory_size_in_bytes":421104,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":3568270,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":18,"memory_in_bytes":133950},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":6781,"deleted":0},"store":{"size_in_bytes":16296415,"throttle_time_in_millis":1465},"indexing":{"index_total":6781,"index_time_in_millis":18053,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":79900,"query_time_in_millis":210164,"query_current":0,"fetch_total":8701,"fetch_time_in_millis":1953,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":63,"total_time_in_millis":3943},"flush":{"total":5,"total_time_in_millis":557},"warmer":{"current":0,"total":68,"total_time_in_millis":198},"filter_cache":{"memory_size_in_bytes":421104,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":3568270,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":18,"memory_in_bytes":133950},"translog":{"operations":0,"size_in_bytes":0}}},"vm-stg_16_2014-10-29_17-36-artworks":{"primaries":{"docs":{"count":2348,"deleted":0},"store":{"size_in_bytes":10005645,"throttle_time_in_millis":0},"indexing":{"index_total":2348,"index_time_in_millis":20124,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":200,"time_in_millis":22,"exists_total":200,"exists_time_in_millis":22,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":13310,"query_time_in_millis":26575,"query_current":0,"fetch_total":4650,"fetch_time_in_millis":637,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":42,"total_time_in_millis":2679},"flush":{"total":5,"total_time_in_millis":226},"warmer":{"current":0,"total":47,"total_time_in_millis":82},"filter_cache":{"memory_size_in_bytes":9431,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":5924691,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":37,"memory_in_bytes":72047},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":2348,"deleted":0},"store":{"size_in_bytes":10005645,"throttle_time_in_millis":0},"indexing":{"index_total":2348,"index_time_in_millis":20124,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":200,"time_in_millis":22,"exists_total":200,"exists_time_in_millis":22,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":13310,"query_time_in_millis":26575,"query_current":0,"fetch_total":4650,"fetch_time_in_millis":637,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":42,"total_time_in_millis":2679},"flush":{"total":5,"total_time_in_millis":226},"warmer":{"current":0,"total":47,"total_time_in_millis":82},"filter_cache":{"memory_size_in_bytes":9431,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":5924691,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":37,"memory_in_bytes":72047},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_3_2014-10-18_20-30-artworks":{"primaries":{"docs":{"count":2154,"deleted":0},"store":{"size_in_bytes":4535425,"throttle_time_in_millis":0},"indexing":{"index_total":2154,"index_time_in_millis":16220,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":20510,"query_time_in_millis":809,"query_current":0,"fetch_total":511,"fetch_time_in_millis":59,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":26,"total_time_in_millis":3600},"flush":{"total":5,"total_time_in_millis":1003},"warmer":{"current":0,"total":31,"total_time_in_millis":12},"filter_cache":{"memory_size_in_bytes":688,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":183270,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":21,"memory_in_bytes":38252},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":2154,"deleted":0},"store":{"size_in_bytes":4535425,"throttle_time_in_millis":0},"indexing":{"index_total":2154,"index_time_in_millis":16220,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":20510,"query_time_in_millis":809,"query_current":0,"fetch_total":511,"fetch_time_in_millis":59,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":26,"total_time_in_millis":3600},"flush":{"total":5,"total_time_in_millis":1003},"warmer":{"current":0,"total":31,"total_time_in_millis":12},"filter_cache":{"memory_size_in_bytes":688,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":183270,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":21,"memory_in_bytes":38252},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_123_2014-10-16_19-04-authors":{"primaries":{"docs":{"count":5198,"deleted":0},"store":{"size_in_bytes":3418642,"throttle_time_in_millis":0},"indexing":{"index_total":5198,"index_time_in_millis":2487,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":0,"query_time_in_millis":0,"query_current":0,"fetch_total":0,"fetch_time_in_millis":0,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":38,"total_time_in_millis":546},"flush":{"total":5,"total_time_in_millis":174},"warmer":{"current":0,"total":43,"total_time_in_millis":29},"filter_cache":{"memory_size_in_bytes":0,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":0,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":33,"memory_in_bytes":63600},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":5198,"deleted":0},"store":{"size_in_bytes":3418642,"throttle_time_in_millis":0},"indexing":{"index_total":5198,"index_time_in_millis":2487,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":0,"query_time_in_millis":0,"query_current":0,"fetch_total":0,"fetch_time_in_millis":0,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":38,"total_time_in_millis":546},"flush":{"total":5,"total_time_in_millis":174},"warmer":{"current":0,"total":43,"total_time_in_millis":29},"filter_cache":{"memory_size_in_bytes":0,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":0,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":33,"memory_in_bytes":63600},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_123_2014-10-16_19-04-artworks":{"primaries":{"docs":{"count":25517,"deleted":0},"store":{"size_in_bytes":110101057,"throttle_time_in_millis":22613},"indexing":{"index_total":25517,"index_time_in_millis":97545,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":4,"time_in_millis":0,"exists_total":4,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":93780,"query_time_in_millis":1157526,"query_current":0,"fetch_total":16946,"fetch_time_in_millis":17213,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":265,"total_time_in_millis":18977},"flush":{"total":5,"total_time_in_millis":4079},"warmer":{"current":0,"total":257,"total_time_in_millis":434},"filter_cache":{"memory_size_in_bytes":2104703,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":8794383,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":37,"memory_in_bytes":624583},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":25517,"deleted":0},"store":{"size_in_bytes":110101057,"throttle_time_in_millis":22613},"indexing":{"index_total":25517,"index_time_in_millis":97545,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":4,"time_in_millis":0,"exists_total":4,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":93780,"query_time_in_millis":1157526,"query_current":0,"fetch_total":16946,"fetch_time_in_millis":17213,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":265,"total_time_in_millis":18977},"flush":{"total":5,"total_time_in_millis":4079},"warmer":{"current":0,"total":257,"total_time_in_millis":434},"filter_cache":{"memory_size_in_bytes":2104703,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":8794383,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":37,"memory_in_bytes":624583},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_18_2014-10-18_18-45b-artworks":{"primaries":{"docs":{"count":10016,"deleted":0},"store":{"size_in_bytes":22417551,"throttle_time_in_millis":1484},"indexing":{"index_total":10016,"index_time_in_millis":16742,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":18787,"time_in_millis":1264,"exists_total":18787,"exists_time_in_millis":1264,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":324330,"query_time_in_millis":2485255,"query_current":0,"fetch_total":56533,"fetch_time_in_millis":7081,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":80,"total_time_in_millis":3436},"flush":{"total":5,"total_time_in_millis":188},"warmer":{"current":0,"total":86,"total_time_in_millis":315},"filter_cache":{"memory_size_in_bytes":758494,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":5433231,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":27,"memory_in_bytes":211797},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":10016,"deleted":0},"store":{"size_in_bytes":22417551,"throttle_time_in_millis":1484},"indexing":{"index_total":10016,"index_time_in_millis":16742,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":18787,"time_in_millis":1264,"exists_total":18787,"exists_time_in_millis":1264,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":324330,"query_time_in_millis":2485255,"query_current":0,"fetch_total":56533,"fetch_time_in_millis":7081,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":80,"total_time_in_millis":3436},"flush":{"total":5,"total_time_in_millis":188},"warmer":{"current":0,"total":86,"total_time_in_millis":315},"filter_cache":{"memory_size_in_bytes":758494,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":5433231,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":27,"memory_in_bytes":211797},"translog":{"operations":0,"size_in_bytes":0}}}}}

Heureusement, des outils existent. Lorsque l’on est développeur, on utilise les bibliothèques de fonction Python ou Ruby voire Java. On fait son petit code et on le lance. Nous, nous sommes des admins, nous préférons la ligne de commande, les « pipes » (tubes) et les enchaînements. Nous allons voir comment jouer avec JSON lorsqu’on est un administrateur.

Pour rappel, JSON est un format de données, présentées sous forme textuelle. Il permet d’échanger des données structurées entre deux programmes. Ce n’est pas un langage de balisage. Les données sont stockées sous forme de « clé/(liste de)valeur(s) », c’est à dire qu’à une clé peut être associée une valeur ou une liste de valeurs.  Pour le reste, la page Wikipédia sur JSON vous aidera.

Je vais illuster les cas d’utilisation en prenant pour exemple un serveur ElasticSearch. Le but n’est pas de présenter ElasticSearch mais l’utlisation de l’outil utilisé pour récupérer des données au format JSON dans ElasticSearch, ElasticSearch servant d’exemple uniquement.

Trentm/json

L’outil que j’utilise se nomme simplement « json« , est disponible sur GitHub et est développé par trentm. Dans la suite et pour éviter la confusion, je nommerai l’outil en « trentm/json » et le format de données en « JSON ». Cet outil requiert l’installation de nodejs. L’installation de nodejs comme l’installation de trentm/json (voir le détail dans la page directement) sont relativement simples et je ne vais pas revenir dessus, veuillez consulter les liens correspondants.

Une fois installé, il suffit de récupérer le contenu JSON et le passer sur la sortie standard de trentm/json, à l’aide des commandes « curl » et « json ». Dans mon cas, pour me faciliter la tâche et votre lecture, je positionne souvent une variable:

cedric@host:~$ export JSONHOST=http://monServeurElasticSearch:9200/

Une fois ceci fait, il suffit de faire :

cedric@host:~$ curl ${JSONHOST}/_stats 2> /dev/null | json

Le résultat est alors affiché de manière plus sympathique :

{
 "_shards": {
 "total": 90,
 "successful": 45,
 "failed": 0
 },
 "_all": {
 "primaries": {
 "docs": {
 "count": 99254,
 "deleted": 0
 },
 "store": {
 "size_in_bytes": 192367723,
 "throttle_time_in_millis": 28623
 },
 "indexing": {
 "index_total": 99254,
 "index_time_in_millis": 218667,
 "index_current": 0,
 "delete_total": 0,
 "delete_time_in_millis": 0,
 "delete_current": 0
 },
...
...
...

Le résultat a été coupé car il est très très très très … long.

Lorsqu’on débute dans l’utilisation d’une API, on regarde la documentation de celle-ci et on fait quelques tests. L’idée est de comprendre le fonctionnement puis de le maitriser petit à petit. On fait quelques appels, on récupère le résultat, on lit la documentation pour connaître le contenu du résultat… Ha? La documentation n’explicite pas clairement le résultat. Comment s’en sortir? Exemple :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json

Le résultat là aussi est un peu long. Pour éviter de devoir tout analyser, nous allons demander uniquement les clés du premier niveau. Cela devrait nous donner une première information. Pour obtenir les clés du premier niveau, il suffit de passer l’option -k (comme keys).

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json -k
[
 "cluster_name",
 "nodes"
]

Nous avons donc deux clés, la première donne le nom du cluster et la seconde les noeuds. Il est à noter (et insister auxprès des développeurs pour cela) qu’il devrait y avoir 0 ou une seule et unique valeur pour « cluster_name » car le nom de la clé est au singulier mais qu’il peut y avoir plusieurs valeurs pour les noeuds car le nom de la clé est au pluriel. Cette notion est très importante et évite aux utilisateurs de l’API d’avoir à se référer constamment à la documentation.

Pour récupérer la valeur de la première clé, il suffit de la passer en argument à l’outil trentm/json (sans l’option -k!) :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json cluster_name
es-prod2

Pour récupérer les valeurs de la seconde clé, l’exercice est identique :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json nodes
{
 "aNBKgBkCSEq9FS7TtjBt6w": {
 "name": "es2.prod",
 "transport_address": "inet[/192.168.1.200:9300]",
 "host": "es2",
 "ip": "192.168.1.200",
 "version": "1.1.1",
 "build": "f1585f0",
 "http_address": "inet[/192.168.1.200:9200]",
 "attributes": {
 "master": "true"
 },
 "settings": {
 "path": {
 "data": "/var/lib/elasticsearch",
 "work": "/tmp/elasticsearch",
 "home": "/usr/share/elasticsearch",
 "conf": "/etc/elasticsearch",
 "logs": "/var/log/elasticsearch"
 },
 "pidfile": "/var/run/elasticsearch.pid",
 "cluster": {
 "name": "es-prod2"
 },

Là encore, le résultat étant un peu long, il a été coupé. Comme il est long, il serait bien d’avoir les clés du premier niveau situées sous le niveau nodes. Intuitivement, on pense qu’il suffit de passer l’option « -k » et en ajoutant « nodes » :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json -k nodes
json: error: cannot use -k|--keys option and lookup arguments together

Malheureusement, c’est impossible. Heureusement, nous sommes des administrateurs intelligents et nous savons qu’une des meilleures inventions du monde est le « pipe« . Il suffit de chaîner les appels pour obtenir les informations souhaitées :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json nodes | json -k
[
 "aNBKgBkCSEq9FS7TtjBt6w"
]

Nous ne disposons que d’un seul nom dont le nom est … incompréhensible. Soit. Tant pis, nous pouvons l’utiliser :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json nodes | json aNBKgBkCSEq9FS7TtjBt6w
{
 "name": "es2.prod",
 "transport_address": "inet[/192.168.1.200:9300]",
 "host": "es2",
 "ip": "192.168.1.200",
 "version": "1.1.1",
 "build": "f1585f0",
 "http_address": "inet[/192.168.1.200:9200]",
 "attributes": {
 "master": "true"
 },
 "settings": {
 "path": {
 "data": "/var/lib/elasticsearch",
 "work": "/tmp/elasticsearch",
 "home": "/usr/share/elasticsearch",
 "conf": "/etc/elasticsearch",
 "logs": "/var/log/elasticsearch"
 },

La solution n’est pas très élégante. Si l’on doit chaîner les appels à la commande json pour chaque niveau, les performances peuvent être fortement impactées lors de l’utilisation de script de maintenance, avec une forte volumétrie et de nombreux appels. Heureusement, il n’est pas nécessaire de faire plusieurs appels à la commande lorsque l’on veut récupérer les données. Il suffit d’utiliser la syntaxe Javascript en ajoutant un point entre les deux clés :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json nodes.aNBKgBkCSEq9FS7TtjBt6w
{
 "name": "es2.prod",
 "transport_address": "inet[/192.168.1.200:9300]",
 "host": "es2",
 "ip": "192.168.1.200",
 "version": "1.1.1",
 "build": "f1585f0",
 "http_address": "inet[/192.168.1.200:9200]",
 "attributes": {
 "master": "true"
 },
 "settings": {
 "path": {
 "data": "/var/lib/elasticsearch",
 "work": "/tmp/elasticsearch",
 "home": "/usr/share/elasticsearch",
 "conf": "/etc/elasticsearch",
 "logs": "/var/log/elasticsearch"
 },

Il est aussi possible de récupérer un 3e niveau en ajoutant un second point et la clé correspondante. Et ainsi de suite :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json nodes.aNBKgBkCSEq9FS7TtjBt6w.name
es2.prod

Il est aussi possible de récupérer plusieurs valeurs en les indiquant les unes à la suite des autres :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json nodes.aNBKgBkCSEq9FS7TtjBt6w.name nodes.aNBKgBkCSEq9FS7TtjBt6w.ip nodes.aNBKgBkCSEq9FS7TtjBt6w.version nodes.aNBKgBkCSEq9FS7TtjBt6w.master
es2.prod
192.168.1.200
1.1.1

Dès lors, on peut optimiser les appels et faciliter le parsing des résultats.

Certaines données renvoyées sont des tableaux. C’est le cas par exemple de nodes.aNBKgBkCSEq9FS7TtjBt6w.jvm.gc_collectors :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json nodes.aNBKgBkCSEq9FS7TtjBt6w.jvm
{
  "pid": 5962,
  "version": "1.7.0_65",
  "vm_name": "OpenJDK 64-Bit Server VM",
  "vm_version": "24.65-b04",
  "vm_vendor": "Oracle Corporation",
  "start_time": 1413556102977,
  "mem": {
    "heap_init_in_bytes": 7516192768,
    "heap_max_in_bytes": 7498760192,
    "non_heap_init_in_bytes": 24313856,
    "non_heap_max_in_bytes": 224395264,
    "direct_max_in_bytes": 7498760192
  },
  "gc_collectors": [
    "ParNew",
    "ConcurrentMarkSweep"
  ],
  "memory_pools": [
    "Code Cache",
    "Par Eden Space",
    "Par Survivor Space",
    "CMS Old Gen",
    "CMS Perm Gen"
  ]
}

Comme on peut le voir, « gc_collectors » comme « memory_pools » sont des tableaux : leurs valeurs sont encadrées par des crochets ouvrants « [«  et fermants « ] », contrairement à « mem » dont les valeurs sont encadrées par des accolades. Pour obtenir la première valeur d’un tableau, il suffit d’y accéder par [0] :

cedric@host:~$ curl ${JSONHOST}/_nodes 2> /dev/null | json nodes.aNBKgBkCSEq9FS7TtjBt6w.jvm.gc_collectors[0] ParNew

Conclusion

JSON est un format souvent utilisé par les développeurs. Ce format est très pratique pour eux, moins pour les administrateurs. Pour utiliser un format utilisé par les développeurs, il est parfois plus simple d’utiliser des outils d’administrateurs (shell, pipe, sed, …), avec un peu de sucre au dessus (trentm/json).

5 réflexions au sujet de « Jouons avec JSON »

  1. Très beau retour sur la manipulation d’un fichier JSON 🙂

    Deux questions :

    (1) Quel est l’utilitaire utilisé pour afficher les données contenues dans un fichier JSON ? J’ai tenté un « apt-get install json » mais ça n’a rien donné et j’hésite sur le paquet à installer parmi les suggestions

    (2) Est-ce que la redirection de la sortie d’erreur dans /dev/null sert à quelque chose dans ce contexte en particulier ? Elles sont affichées à l’écran en cas de pépin sinon ?

    Merci

Laisser un commentaire

Votre adresse de messagerie ne sera pas publiée. Les champs obligatoires sont indiqués avec *