Jouons avec JSON

 

Le très gros point fort du JSON est qu’il est très peu verbeux (contrairement au XML) et performant à analyser (contrairement au …). C’est aussi un point faible pour nous administrateurs, humains, qui avons du mal à le comprendre. Pour cela, il faut pouvoir le lire. Ce n’est pas toujours évident, comme le montre cet exemple :

{"_shards":{"total":90,"successful":45,"failed":0},"_all":{"primaries":{"docs":{"count":99254,"deleted":0},"store":{"size_in_bytes":192367723,"throttle_time_in_millis":28623},"indexing":{"index_total":99254,"index_time_in_millis":218667,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":24550,"time_in_millis":2000,"exists_total":24534,"exists_time_in_millis":2000,"missing_total":16,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":993640,"query_time_in_millis":4358463,"query_current":0,"fetch_total":197801,"fetch_time_in_millis":40594,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":773,"total_time_in_millis":43183},"flush":{"total":50,"total_time_in_millis":12412},"warmer":{"current":0,"total":822,"total_time_in_millis":1275},"filter_cache":{"memory_size_in_bytes":3495180,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":32400201,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":238,"memory_in_bytes":1600228},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":99254,"deleted":0},"store":{"size_in_bytes":192367723,"throttle_time_in_millis":28623},"indexing":{"index_total":99254,"index_time_in_millis":218667,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":24550,"time_in_millis":2000,"exists_total":24534,"exists_time_in_millis":2000,"missing_total":16,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":993640,"query_time_in_millis":4358463,"query_current":0,"fetch_total":197801,"fetch_time_in_millis":40594,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":773,"total_time_in_millis":43183},"flush":{"total":50,"total_time_in_millis":12412},"warmer":{"current":0,"total":822,"total_time_in_millis":1275},"filter_cache":{"memory_size_in_bytes":3495180,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":32400201,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":238,"memory_in_bytes":1600228},"translog":{"operations":0,"size_in_bytes":0}}},"indices":{"vm-dev_123_2014-10-16_19-04-medias":{"primaries":{"docs":{"count":43053,"deleted":0},"store":{"size_in_bytes":9326582,"throttle_time_in_millis":3059},"indexing":{"index_total":43053,"index_time_in_millis":13411,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":0,"query_time_in_millis":0,"query_current":0,"fetch_total":0,"fetch_time_in_millis":0,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":199,"total_time_in_millis":2249},"flush":{"total":10,"total_time_in_millis":5592},"warmer":{"current":0,"total":220,"total_time_in_millis":119},"filter_cache":{"memory_size_in_bytes":0,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":0,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":15,"memory_in_bytes":338520},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":43053,"deleted":0},"store":{"size_in_bytes":9326582,"throttle_time_in_millis":3059},"indexing":{"index_total":43053,"index_time_in_millis":13411,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":0,"query_time_in_millis":0,"query_current":0,"fetch_total":0,"fetch_time_in_millis":0,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":199,"total_time_in_millis":2249},"flush":{"total":10,"total_time_in_millis":5592},"warmer":{"current":0,"total":220,"total_time_in_millis":119},"filter_cache":{"memory_size_in_bytes":0,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":0,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":15,"memory_in_bytes":338520},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_16_2014-10-21_12-17-artworks":{"primaries":{"docs":{"count":2348,"deleted":0},"store":{"size_in_bytes":9316186,"throttle_time_in_millis":0},"indexing":{"index_total":2348,"index_time_in_millis":7756,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":4702,"time_in_millis":678,"exists_total":4686,"exists_time_in_millis":678,"missing_total":16,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":407110,"query_time_in_millis":429377,"query_current":0,"fetch_total":106584,"fetch_time_in_millis":13256,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":32,"total_time_in_millis":731},"flush":{"total":5,"total_time_in_millis":219},"warmer":{"current":0,"total":37,"total_time_in_millis":13},"filter_cache":{"memory_size_in_bytes":99792,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":4375465,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":27,"memory_in_bytes":63977},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":2348,"deleted":0},"store":{"size_in_bytes":9316186,"throttle_time_in_millis":0},"indexing":{"index_total":2348,"index_time_in_millis":7756,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":4702,"time_in_millis":678,"exists_total":4686,"exists_time_in_millis":678,"missing_total":16,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":407110,"query_time_in_millis":429377,"query_current":0,"fetch_total":106584,"fetch_time_in_millis":13256,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":32,"total_time_in_millis":731},"flush":{"total":5,"total_time_in_millis":219},"warmer":{"current":0,"total":37,"total_time_in_millis":13},"filter_cache":{"memory_size_in_bytes":99792,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":4375465,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":27,"memory_in_bytes":63977},"translog":{"operations":0,"size_in_bytes":0}}},"vm-stg_29_2014-10-23_17-34-artworks":{"primaries":{"docs":{"count":1839,"deleted":0},"store":{"size_in_bytes":6950220,"throttle_time_in_millis":0},"indexing":{"index_total":1839,"index_time_in_millis":26329,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":857,"time_in_millis":36,"exists_total":857,"exists_time_in_millis":36,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":54700,"query_time_in_millis":48757,"query_current":0,"fetch_total":3876,"fetch_time_in_millis":395,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":28,"total_time_in_millis":7022},"flush":{"total":5,"total_time_in_millis":374},"warmer":{"current":0,"total":33,"total_time_in_millis":73},"filter_cache":{"memory_size_in_bytes":100968,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":4120891,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":23,"memory_in_bytes":53502},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":1839,"deleted":0},"store":{"size_in_bytes":6950220,"throttle_time_in_millis":0},"indexing":{"index_total":1839,"index_time_in_millis":26329,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":857,"time_in_millis":36,"exists_total":857,"exists_time_in_millis":36,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":54700,"query_time_in_millis":48757,"query_current":0,"fetch_total":3876,"fetch_time_in_millis":395,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":28,"total_time_in_millis":7022},"flush":{"total":5,"total_time_in_millis":374},"warmer":{"current":0,"total":33,"total_time_in_millis":73},"filter_cache":{"memory_size_in_bytes":100968,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":4120891,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":23,"memory_in_bytes":53502},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_28_2014-10-18_19-43-artworks":{"primaries":{"docs":{"count":6781,"deleted":0},"store":{"size_in_bytes":16296415,"throttle_time_in_millis":1465},"indexing":{"index_total":6781,"index_time_in_millis":18053,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":79900,"query_time_in_millis":210164,"query_current":0,"fetch_total":8701,"fetch_time_in_millis":1953,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":63,"total_time_in_millis":3943},"flush":{"total":5,"total_time_in_millis":557},"warmer":{"current":0,"total":68,"total_time_in_millis":198},"filter_cache":{"memory_size_in_bytes":421104,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":3568270,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":18,"memory_in_bytes":133950},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":6781,"deleted":0},"store":{"size_in_bytes":16296415,"throttle_time_in_millis":1465},"indexing":{"index_total":6781,"index_time_in_millis":18053,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":79900,"query_time_in_millis":210164,"query_current":0,"fetch_total":8701,"fetch_time_in_millis":1953,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":63,"total_time_in_millis":3943},"flush":{"total":5,"total_time_in_millis":557},"warmer":{"current":0,"total":68,"total_time_in_millis":198},"filter_cache":{"memory_size_in_bytes":421104,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":3568270,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":18,"memory_in_bytes":133950},"translog":{"operations":0,"size_in_bytes":0}}},"vm-stg_16_2014-10-29_17-36-artworks":{"primaries":{"docs":{"count":2348,"deleted":0},"store":{"size_in_bytes":10005645,"throttle_time_in_millis":0},"indexing":{"index_total":2348,"index_time_in_millis":20124,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":200,"time_in_millis":22,"exists_total":200,"exists_time_in_millis":22,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":13310,"query_time_in_millis":26575,"query_current":0,"fetch_total":4650,"fetch_time_in_millis":637,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":42,"total_time_in_millis":2679},"flush":{"total":5,"total_time_in_millis":226},"warmer":{"current":0,"total":47,"total_time_in_millis":82},"filter_cache":{"memory_size_in_bytes":9431,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":5924691,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":37,"memory_in_bytes":72047},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":2348,"deleted":0},"store":{"size_in_bytes":10005645,"throttle_time_in_millis":0},"indexing":{"index_total":2348,"index_time_in_millis":20124,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":200,"time_in_millis":22,"exists_total":200,"exists_time_in_millis":22,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":13310,"query_time_in_millis":26575,"query_current":0,"fetch_total":4650,"fetch_time_in_millis":637,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":42,"total_time_in_millis":2679},"flush":{"total":5,"total_time_in_millis":226},"warmer":{"current":0,"total":47,"total_time_in_millis":82},"filter_cache":{"memory_size_in_bytes":9431,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":5924691,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":37,"memory_in_bytes":72047},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_3_2014-10-18_20-30-artworks":{"primaries":{"docs":{"count":2154,"deleted":0},"store":{"size_in_bytes":4535425,"throttle_time_in_millis":0},"indexing":{"index_total":2154,"index_time_in_millis":16220,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":20510,"query_time_in_millis":809,"query_current":0,"fetch_total":511,"fetch_time_in_millis":59,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":26,"total_time_in_millis":3600},"flush":{"total":5,"total_time_in_millis":1003},"warmer":{"current":0,"total":31,"total_time_in_millis":12},"filter_cache":{"memory_size_in_bytes":688,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":183270,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":21,"memory_in_bytes":38252},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":2154,"deleted":0},"store":{"size_in_bytes":4535425,"throttle_time_in_millis":0},"indexing":{"index_total":2154,"index_time_in_millis":16220,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":20510,"query_time_in_millis":809,"query_current":0,"fetch_total":511,"fetch_time_in_millis":59,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":26,"total_time_in_millis":3600},"flush":{"total":5,"total_time_in_millis":1003},"warmer":{"current":0,"total":31,"total_time_in_millis":12},"filter_cache":{"memory_size_in_bytes":688,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":183270,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":21,"memory_in_bytes":38252},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_123_2014-10-16_19-04-authors":{"primaries":{"docs":{"count":5198,"deleted":0},"store":{"size_in_bytes":3418642,"throttle_time_in_millis":0},"indexing":{"index_total":5198,"index_time_in_millis":2487,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":0,"query_time_in_millis":0,"query_current":0,"fetch_total":0,"fetch_time_in_millis":0,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":38,"total_time_in_millis":546},"flush":{"total":5,"total_time_in_millis":174},"warmer":{"current":0,"total":43,"total_time_in_millis":29},"filter_cache":{"memory_size_in_bytes":0,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":0,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":33,"memory_in_bytes":63600},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":5198,"deleted":0},"store":{"size_in_bytes":3418642,"throttle_time_in_millis":0},"indexing":{"index_total":5198,"index_time_in_millis":2487,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":0,"time_in_millis":0,"exists_total":0,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":0,"query_time_in_millis":0,"query_current":0,"fetch_total":0,"fetch_time_in_millis":0,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":38,"total_time_in_millis":546},"flush":{"total":5,"total_time_in_millis":174},"warmer":{"current":0,"total":43,"total_time_in_millis":29},"filter_cache":{"memory_size_in_bytes":0,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":0,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":33,"memory_in_bytes":63600},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_123_2014-10-16_19-04-artworks":{"primaries":{"docs":{"count":25517,"deleted":0},"store":{"size_in_bytes":110101057,"throttle_time_in_millis":22613},"indexing":{"index_total":25517,"index_time_in_millis":97545,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":4,"time_in_millis":0,"exists_total":4,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":93780,"query_time_in_millis":1157526,"query_current":0,"fetch_total":16946,"fetch_time_in_millis":17213,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":265,"total_time_in_millis":18977},"flush":{"total":5,"total_time_in_millis":4079},"warmer":{"current":0,"total":257,"total_time_in_millis":434},"filter_cache":{"memory_size_in_bytes":2104703,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":8794383,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":37,"memory_in_bytes":624583},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":25517,"deleted":0},"store":{"size_in_bytes":110101057,"throttle_time_in_millis":22613},"indexing":{"index_total":25517,"index_time_in_millis":97545,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":4,"time_in_millis":0,"exists_total":4,"exists_time_in_millis":0,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":93780,"query_time_in_millis":1157526,"query_current":0,"fetch_total":16946,"fetch_time_in_millis":17213,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":265,"total_time_in_millis":18977},"flush":{"total":5,"total_time_in_millis":4079},"warmer":{"current":0,"total":257,"total_time_in_millis":434},"filter_cache":{"memory_size_in_bytes":2104703,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":8794383,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":37,"memory_in_bytes":624583},"translog":{"operations":0,"size_in_bytes":0}}},"vm-dev_18_2014-10-18_18-45b-artworks":{"primaries":{"docs":{"count":10016,"deleted":0},"store":{"size_in_bytes":22417551,"throttle_time_in_millis":1484},"indexing":{"index_total":10016,"index_time_in_millis":16742,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":18787,"time_in_millis":1264,"exists_total":18787,"exists_time_in_millis":1264,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":324330,"query_time_in_millis":2485255,"query_current":0,"fetch_total":56533,"fetch_time_in_millis":7081,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":80,"total_time_in_millis":3436},"flush":{"total":5,"total_time_in_millis":188},"warmer":{"current":0,"total":86,"total_time_in_millis":315},"filter_cache":{"memory_size_in_bytes":758494,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":5433231,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":27,"memory_in_bytes":211797},"translog":{"operations":0,"size_in_bytes":0}},"total":{"docs":{"count":10016,"deleted":0},"store":{"size_in_bytes":22417551,"throttle_time_in_millis":1484},"indexing":{"index_total":10016,"index_time_in_millis":16742,"index_current":0,"delete_total":0,"delete_time_in_millis":0,"delete_current":0},"get":{"total":18787,"time_in_millis":1264,"exists_total":18787,"exists_time_in_millis":1264,"missing_total":0,"missing_time_in_millis":0,"current":0},"search":{"open_contexts":0,"query_total":324330,"query_time_in_millis":2485255,"query_current":0,"fetch_total":56533,"fetch_time_in_millis":7081,"fetch_current":0},"merges":{"current":0,"current_docs":0,"current_size_in_bytes":0,"total":0,"total_time_in_millis":0,"total_docs":0,"total_size_in_bytes":0},"refresh":{"total":80,"total_time_in_millis":3436},"flush":{"total":5,"total_time_in_millis":188},"warmer":{"current":0,"total":86,"total_time_in_millis":315},"filter_cache":{"memory_size_in_bytes":758494,"evictions":0},"id_cache":{"memory_size_in_bytes":0},"fielddata":{"memory_size_in_bytes":5433231,"evictions":0},"percolate":{"total":0,"time_in_millis":0,"current":0,"memory_size_in_bytes":-1,"memory_size":"-1b","queries":0},"completion":{"size_in_bytes":0},"segments":{"count":27,"memory_in_bytes":211797},"translog":{"operations":0,"size_in_bytes":0}}}}}

Heureusement, des outils existent. Lorsque l’on est développeur, on utilise les bibliothèques de fonction Python ou Ruby voire Java. On fait son petit code et on le lance. Nous, nous sommes des admins, nous préférons la ligne de commande, les « pipes » (tubes) et les enchaînements. Nous allons voir comment jouer avec JSON lorsqu’on est un administrateur.

Continuer la lecture de Jouons avec JSON

Optimisation d’un script SHELL

 

Nous avons un script shell. Celui-ci fonctionne en deux étapes. La première étape dure une nuit. La deuxième étape dure elle aussi toute une nuit. Cependant, ce processus doit être exécuté pour 60 serveurs. La durée totale théorique est donc de 120 jours!

La première étape peut être parallélisée, sur les 60 serveurs en même temps. La première étape peut donc avoir une durée d’une nuit si on lance le processus sur les 60 serveurs en parallèle. La deuxième étape ne peut être parallélisée car elle doit être exécutée sur un seul et unique serveur et elle génère 60 résultats différents (un par serveur source, dépendant du résultat de l’étape 1 des serveurs sources). La 2e étape a donc une durée incompressible de 60 nuits.

Il faut donc au moins 61 nuits (1 + 60) pour exécuter l’intégralité du processus. Trop long. Trop complexe. Trop peu sûr. Comment optimiser cela?

Edit : précisions apportées suite au commentaire de « bob morane », merci à lui!

Continuer la lecture de Optimisation d’un script SHELL

Optimiser la connexion SSH à des serveurs distants

(Image « En plein taff.. » sous licence CC-BY-2.0 par  Thomassin Mickaël)

Le télétravail est une belle invention. Vous pouvez travailler de chez vous, commencer plus tôt et terminer plus tard tout en étant moins fatigué. Vous disposez d’une certaine souplesse, après vous être mis d’accord avec votre responsable, pour vous absenter une petite heure pour aller conduite la voiture au garage par exemple. Vous évitez les transports en commun, très aléatoire en région parisienne, ou les 10 petits kilomètres en voiture à une vitesse moyenne de… 20km/h. Bref, c’est que du bonheur…

Vous lancez votre VPN et vous vous connectez en SSH sur votre poste. De là, vous pouvez lancer des applications graphiques qui s’exécutent sur votre poste au bureau mais s’affichent sur votre ordinateur familial. Vous disposez d’une connexion supra-giga-ultra-méga-très-très-haut débit, grâce à la fibre optique, c’est à dire du 100MBits/s (!!! vivement le supra-giga-ultra-méga-très-très-haut ++ débit) en download et du 50MBits/s en upload. Petit veinard! En dehors du fait qu’on se moque de nous sur le côté très-très-très-haut débit, on constate quand même que ça reste assez… lent. Il y a une certaine latence entre la frappe d’une touche sur le clavier et le moment où le caractère s’affiche. De plus, Firefox et LibreOffice mettent un certain temps à s’afficher lors de nombreux « Alt-Tab ». Ceci est très gênant. D’autant plus gênant lorsqu’on enchaîne deux connexions SSH dans deux tunnels OpenVPN distincts pour lancer un Firefox à l’autre bout afin d’utiliser l’outil d’administration graphique du SAN ou de l’outil de virtualisation. Il peut se passer plusieurs dizaines de secondes entre le changement de fenêtre et le rafraîchissement de l’affichage de cette fenêtre. Comment « accélérer » la connexion SSH? Comment améliorer le ressenti utilisateur?

Continuer la lecture de Optimiser la connexion SSH à des serveurs distants

Statistiques 2014 du blog

 

Un petit retour en arrière peut faire beaucoup de bien. C’est l’occasion de voir ce qui a fonctionné et surtout ce qui n’a pas fonctionné. Riche d’enseignements, ces statistiques de consultation me permettent d’apprendre beaucoup sur moi mais aussi sur les centres d’intérêts des personnes qui visitent le blog. Je peux alors adapter le contenu aux souhaits (supposés!) de mes lecteurs. Bien entendu, m’adapter ne veut pas dire « pervertir mon âme » : je ne vais pas parler d’iPhone quand bien même un millions de personnes me le demanderais :-). Quelles sont les enseignements de cette année 2014?

Continuer la lecture de Statistiques 2014 du blog

Weblog de Cédric Temple is Stephen Fry proof thanks to caching by WP Super Cache