================================================== Changes from 2017-01-02 00:00:00 +0000 to present. ================================================== ---------------------------------------- version 6.4 at 2019-01-02 22:25:09 +0000 ---------------------------------------- Change: 470feb7d6e00998906849cbfb49d1904eb79cf6b Author: Brad Lhotsky Date : 2019-01-02 14:20:36 +0000 Updated Documentation for the 6.4 release Change: 095da97cc9e9fb29ebcea13851d548e20c6166d6 Author: Brad Lhotsky Date : 2019-01-02 13:40:05 +0000 Add in the new QueryString Extensions to the documentation Change: 714c9856d382bcc1ee95fc2ecd8ee7257e13aaec Author: Brad Lhotsky Date : 2019-01-02 12:12:56 +0000 Scroll search corrected to use the correct API Finally migrated off the 0.x version of the scrolled search API which is no longer supported in 6.5. Change: 4a9e798426a21f2ab3543317bea5a56937536b56 Author: Brad Lhotsky Date : 2019-01-02 11:18:28 +0000 Convert ::QueryString::AutoEscape to use terms filter Rather than attempting to escape characters, use the Elastic `terms` filter to perform the lookup if the user specifies `=field:value`. This will convert to: { terms: { field: 'value' } } And that will perform an exact match to the documents in a more predictable and future-proof manner. ---------------------------------------- version 6.3 at 2019-01-01 01:00:09 +0000 ---------------------------------------- Change: 0a032698e44212328cb3bef1acae61812db12353 Author: Brad Lhotsky Date : 2018-12-31 17:00:09 +0000 Fixes for the 6.x version of ElasticSearch * Ensure we set a Content-Type header for requests sending JSON * Handle special casing for index creation in ::Connection * Ensure all _bulk requests end with a "\n" * Fix missing Ref::Util import in es-copy-index.pl * es-copy-index.pl: only send _mappings if defined Change: 4683229bdb8811d7d1122f4a9bd665c61d84f23d Author: Brad Lhotsky Date : 2018-12-31 16:49:20 +0000 Fix authordeps Change: 75b43dee4c8313e16891e2bb7bfe8e44b99593a9 Author: Brad Lhotsky Date : 2018-12-31 16:45:25 +0000 Reset author/copyright_holder because my git settings trip up the auto-discovery. Change: 510aa532e05a9c2cb44af384145c2beaa06c65e2 Author: Brad Lhotsky Date : 2018-12-27 22:37:01 +0000 Fix up the AutoEscape plugin to run at the correct priority Change: 90d61d84d2c2ece75b7bc5d13690ca1b282c7ad7 Author: Brad Lhotsky Date : 2018-12-27 22:20:31 +0000 Added an AutoEscape Query String Parser While searching for UserAgents or file paths, you need to know the secret Lucene handshakes to match as you expect. This plugin automatically converts common characters users may not expect to cause special behavior. ---------------------------------------- version 6.2 at 2018-12-21 02:53:52 +0000 ---------------------------------------- Change: 81bae677ac93da9eba1c2a3b4859eded645782f9 Author: Brad Lhotsky Date : 2018-12-20 18:53:52 +0000 Documentation and release fixes * Removed Test::Coverage::TrustPod and Test::Critic from requires and moved them to the authordeps sections * Fixed Running of Extra Tests on Release * Added a perlcritic.rc Change: 0eab9bf3babfb37064e4af13c579e42013a45057 Author: Brad Lhotsky Date : 2018-12-20 18:37:13 +0000 Add 'jsonpretty' as a format When using `--format 'json'`, default to setting `--no-decorators` since we'll mostly use this with `jq`. Change: 1f6a51a1cca972d441fc9aca2dfd3169c815a87f Author: Brad Lhotsky Date : 2018-12-20 18:21:56 +0000 Output tweaks for better usability * Added `--no-decorators` option as an override for `--no-header` * When using `--no-decorators`, disable outputting the footers as well * Only display summary information when we're dealing with multi-day output * When an agg is `terms` or `significant_terms` and no other collision exists, suppress the `$type` prefix to be prettier Change: 1a8a5394e2c99f4cf59d670d36705ce06b41ae92 Author: Brad Lhotsky Date : 2018-09-26 17:28:02 +0000 Allow the user to change the max-batch-size/scroll size Change: 298902bf52642fcc081059f9f85853ba7449d865 Author: Mohammad S Anwar Date : 2018-07-21 10:00:45 +0000 Autogenerate META.yml as suggested by CPANTS. ---------------------------------------- version 6.1 at 2018-09-18 03:10:27 +0000 ---------------------------------------- Change: 020fddc36e35230b1a275c87c8edddae2fbb7c7b Author: Brad Lhotsky Date : 2018-09-17 20:10:27 +0000 Updates for the documentation for 6.1 release Change: 263e77f276c95dbf1f90bba6b77264547e4b7390 Author: Brad Lhotsky Date : 2018-09-17 20:09:05 +0000 Add --bg-filter option for es-search.pl If significant_terms is used for the --top aggregation, this option is able to inject a background_filter into the significant_terms aggregation. Change: e76897d8eefee23d70b9fd5b3e6bd26bd7494b95 Author: Brad Lhotsky Date : 2018-09-17 20:08:24 +0000 Update to using GitHub::Meta and rearrange Pod::Weaver configs ---------------------------------------- version 6.0 at 2018-07-28 00:45:25 +0000 ---------------------------------------- Change: e58dcf2f190a1db3e1e10f8799b67c7f9a8abcc4 Author: Brad Lhotsky Date : 2018-07-27 17:45:25 +0000 Fix a bug in conditional checks for index age when no date is found in the index name Change: f6e05fcf90fa575f5e5b5ccafb0bf94667ef9bd5 Author: Brad Lhotsky Date : 2018-07-23 14:31:41 +0000 New es_local_index_meta() function to allow looking up the timestamp field in es-search.pl much easier and more straight-forward. ---------------------------------------- version 5.9 at 2018-07-21 01:10:38 +0000 ---------------------------------------- Change: 9b00c64735a21b74223fddcde44fff74772efd17 Author: Brad Lhotsky Date : 2018-07-20 18:10:38 +0000 Bumping Release to 5.9 Change: 415c84da87571aa488a0f246d423e24fbf446e00 Author: Brad Lhotsky Date : 2018-07-18 17:58:55 +0000 Fix another uninitialized error in the code ---------------------------------------- version 5.8 at 2018-07-16 19:14:45 +0000 ---------------------------------------- Change: 3f276cdc5e5b8a1fc6215275b8b3c24e3d8ef066 Author: Brad Lhotsky Date : 2018-07-16 12:14:45 +0000 Docs update for 5.8 release Change: 12336ccb1ee03eaa294af46a1340af3cbb47f4c1 Author: Brad Lhotsky Date : 2018-07-16 11:45:58 +0000 Fix a warning and improve output * Fix a "use of uninitialized" warning for when an index is missing a date string but matches the base pattern * Fix logic bug that meant '--delete --delete-days 1' was a noop because of using '<' instead of '<=' Change: e3b0899af37835e7e9d372788c31339cc3cf8f01 Author: Brad Lhotsky Date : 2018-07-16 11:43:38 +0000 Bubble up errors so we know why we failed Change: 9a07d69d5a7571d58951682c3d190083cf0a84df Author: Brad Lhotsky Date : 2018-07-16 11:25:16 +0000 Show shards breakdown by primary and replica ---------------------------------------- version 5.7 at 2018-07-06 05:34:05 +0000 ---------------------------------------- Change: e90729b13bb4fc45d68e9711ac139c8f8ab1dad8 Author: Brad Lhotsky Date : 2018-07-05 22:34:05 +0000 Documentation Updates for 5.7 release Change: 7498a4ccea72cd4cc8860316bf3aa175e278c158 Author: Brad Lhotsky Date : 2018-07-05 22:23:55 +0000 Add support for statistical aggregations via --with Change: 8a86345e449b355257358de7f81414d278442a38 Author: Brad Lhotsky Date : 2018-07-05 22:05:43 +0000 Fix joins when using "NOT" for negations Change: 4e103a8feb362f9b5722736c3fdda66d9f36819a Author: Brad Lhotsky Date : 2018-07-05 20:20:40 +0000 Fixup broken query strings without joining keywords This addresses the most used case where a query_string is missing the keyword 'AND' in the token joining. This also makes it easier to produce longer queries with less thinking. Change: 1b43ea3120ae6ff51bc9f0d7c04e46f05499e7bb Author: Brad Lhotsky Date : 2018-07-02 23:13:54 +0000 Fix documentation for the ::Ranges plugin Change: c7cb7c238a01c235823161c7d4455afe00665870 Author: Brad Lhotsky Date : 2018-07-02 22:58:21 +0000 Add a --filter option to es-search.pl Make use of the newly exposed 'filter' context from ::QueryString and add positive matches as either 'must' (query context) or 'filter' (filter context). Change: 498af1d1759cf2788a06526a847d331e81e028d6 Author: Brad Lhotsky Date : 2018-07-02 22:55:00 +0000 Allow toggling the context between 'query' and 'filter' By overriding the ::QueryString context, it's possible to toggle between collecting the positive matches as 'filter' elements instead of 'must'. This allows the use of unscored queries which may improve performance for some queries dramatically. Change: 601bb7eea49cc3a8dbe0c9579c6a0b94e5675824 Author: Brad Lhotsky Date : 2018-07-02 22:53:07 +0000 Early return for ::QueryString:: plugins Both the ::Nested and ::Ranges plugins would emit uninitialized warnings when run due to forgetting to check if the token matched the plugin. Change: 67342d859955cc6f4ca4b6a75363beb08f7af1d8 Author: Brad Lhotsky Date : 2018-07-02 21:41:22 +0000 Fix parameter specification in the library too Change: 1039688ce0316e52f73f3ff57ec7b2ddedf0636b Author: Brad Lhotsky Date : 2018-07-02 21:36:26 +0000 Fix the parameter specifications and remove unsupported script Change: 60aaa2dc1ea4ad19f0d5837a3620e9f98029196e Author: Brad Lhotsky Date : 2018-07-02 21:21:54 +0000 Add ::QueryString::Ranges Translates queries with field:<,<=,>,>-= to equivalent range queries Change: ca4e73f856bd9f48818e3fd2a7b959998dee8047 Author: Brad Lhotsky Date : 2018-07-02 08:58:05 +0000 Fix --base in es-search.pl, make --index less policey. * --base was broken as es-search.pl is looking for open indices. Prepend the wildcard to the '/_stats/docs' request. * Don't validate the --index parameter, instead pass it through. This gives the user the power to create their own index strings Change: a880e6ff664b99e64da8d18589f66bc51f1e089c Author: Brad Lhotsky Date : 2018-07-02 08:58:05 +0000 Replace es-storage-data.pl with es-storage-overview.pl Change: 7bb0c293fd3f433183002c75be6a7e0e6e0f53fc Author: Brad Lhotsky Date : 2018-06-18 20:49:18 +0000 Remove size parameter if set to zero. For statistical aggregations, the size parameter is invalid. This is a shortcut to remove the parameter from the sub aggregation. Change: 1e6372612b5e86627f6df2703c8a49e344bd129d Author: Brad Lhotsky Date : 2018-06-08 16:13:52 +0000 Fix failing tests Change: 3053cd3ee7c1f30138f5fe2f51b6e77dc6639bba Author: Brad Lhotsky Date : 2018-05-10 21:40:51 +0000 Make the fast things fast * Reorder the --bases for es-search.pl to skip index meta checks. * Use es_indices(check_date => 0) instead of _all => 1 to skip closed indices. * Make es_index_bases() simpler, i.e.: Old behavior: (a, a-b, a-b-c, b-c, c) = es_index_bases('a-b-c'); New behavior: (a, a-b, a-b-c) = es_index_bases('a-b-c'); * Use _stats/docs to resolve all index names as indexes without explicit aliases don't show up in all versions of ElasticSearch. Change: ce08b1c0cc3c9432780612d13dd1214dc60be04e Author: Brad Lhotsky Date : 2018-05-10 21:06:34 +0000 New script: es-storage-overview.pl Implements the simplest possible overview of storage. Does not provide node or index data, but gives a high level overview. Change: 3837060be0e0145b2e23987da6fab11af9d54a4e Author: Brad Lhotsky Date : 2018-05-10 20:29:24 +0000 es-search.pl - more concise output When we encounter a record that is an array, but is holding only 1 item, derefernce and extract the single value. Change: b2473b3b412444a8148dfd96b0a07eb0dc466054 Author: Brad Lhotsky Date : 2018-04-30 19:57:37 +0000 Made es_request() errors fatal and sped up index lookup ---------------------------------------- version 5.6 at 2018-04-13 18:20:03 +0000 ---------------------------------------- Change: fcb981c7ea59b5e3ac5f9c5b2ae3cf34e1fad5a3 Author: Brad Lhotsky Date : 2018-04-13 20:20:03 +0000 Documentation fixup with the latest pod tooling Change: 2d9fce9813a585ff3115bfdce4b6b46a09192bf7 Author: Brad Lhotsky Date : 2018-04-13 10:57:03 +0000 Don't track the local TODO file Change: 7727fd81af17556c54e9c911d507efcd20266765 Author: Brad Lhotsky Date : 2018-04-13 10:54:44 +0000 Ready for 5.6 Release * Remove overzealous debugging messages * Tweak es-storage-data.pl to provide full details via --all * Change the way es-daily-index-maintenance.pl complains when --master--only is set. * Documentation updated Change: 1b06f8ce6a574e34acea51938193d7590c6c24f9 Author: Brad Lhotsky Date : 2018-03-17 00:49:08 +0000 Docs updated added --view base to es-storage-data.pl Change: 0cf82d70c23104b590f29b4551612578541cbb6a Author: Brad Lhotsky Date : 2018-03-17 00:39:18 +0000 Get es-storage-data.pl working on modern ElasticSearch ---------------------------------------- version 5.5 at 2018-03-17 06:01:25 +0000 ---------------------------------------- Change: 757dec9e648d077759da9a19ebdf3ed343df82f1 Author: Brad Lhotsky Date : 2018-03-16 23:01:25 +0000 Remove no longer necessary test Change: d63a1b823fcd72116e731cbfc45387290b60e769 Author: Brad Lhotsky Date : 2018-03-16 22:49:15 +0000 Documentation updated for 5.5 release Change: b1709e428bf9031e99b196ed6c865b02535ddb66 Author: Brad Lhotsky Date : 2018-03-16 22:43:04 +0000 Bugfix and better replica management * BUGFIX: es_index_strip_date() now removes everything after the date string. es-daily-index-maintenance.pl * --replicas-* options aliased to --replica-* * Smarter handling of math around replica calculation * Disable --bloom option entirely as it's deprecated by Elastic Change: aabba5ad9a1c26ee5aa631f4ea81d887bf98463a Author: Brad Lhotsky Date : 2018-02-21 23:04:18 +0000 Pod and Perl::Critic fixes Change: 1dcaaa7ea8b7b8b89387692dd969a18fd385bd23 Author: Brad Lhotsky Date : 2018-02-21 20:58:15 +0000 Conversion to Ref::Util and Types::Tiny Fix paths for the shebang line so the installer will be able to install the scripts with the correct Perl binary path. Add a first pass at nested query syntax Change: 722e812a7365f9a81aa0d1df5d61a3a1dcb695ed Author: Brad Lhotsky Date : 2018-02-21 20:52:06 +0000 Update the QS::IP expander to make it more intelligent. If a field ends in *_ip, allow CIDR queries against the field. This skips renaming the field from "src_ip" to "src_ip_numeric" and is now an inline replacement. This enhances simplifies the CLI queries. The docs were not accurate, so now the example and documentation are valid. Change: 8e7e34fe1690349a0485ec271c85e966c699b0d9 Author: Brad Lhotsky Date : 2017-12-03 23:42:00 +0000 Disallow tabs in my files. Change: 938d5f9e9bf387d304388bf524348dd4b0d51fac Author: Brad Lhotsky Date : 2017-10-25 17:07:47 +0000 Allow --show to be more flexible * Unbind from the index fields since this operates on the source document * Change to an array so you can add multiple --show's ---------------------------------------- version 5.4 at 2017-10-13 19:36:20 +0000 ---------------------------------------- Change: c31b5f39953589c497f4374be9c22f36a2514dc2 Author: Brad Lhotsky Date : 2017-10-13 12:36:20 +0000 Documentation update for the 5.4 release. Change: 3b6a379937869eaf46e80eea6f7d54e5c4f2220a Author: Brad Lhotsky Date : 2017-07-12 20:50:57 +0000 Default to UTC, warn when using incompatible options. Change: 570de7b2ffa8896b723a3d5c739cec25f4fe8297 Author: Lisa Hare Date : 2017-06-23 08:50:42 +0000 add older option to es-apply-settings when older argument is passed to es_indices in Utilities.pm if days is also specified then it will only return indexes older that the specified days Refs #24 Change: 04a72068727474471abd709e993b9095bcf4e489 Author: Brad Lhotsky Date : 2017-06-07 11:05:12 +0000 Fixup the optimize call for newer ElasticSearch releases. Change: 25ad9e95697a92093c30375836041c7fb2f691e8 Author: Markus Linnala Date : 2017-04-05 19:33:52 +0000 es-alias-manager.pl: relax index name checker to allow _ ---------------------------------------- version 5.3 at 2017-03-23 01:49:52 +0000 ---------------------------------------- Change: 3becaead7ab98d420511e67ac55724eb19c3bd76 Author: Brad Lhotsky Date : 2017-03-22 18:49:52 +0000 Version 5.3 Docs Change: 2fb6c954347fbbd3a5c8dc372effe8e84116b908 Author: Brad Lhotsky Date : 2017-03-22 18:48:01 +0000 ElasticSearch 5.2.x support and enhancements * Elasticsearch no longer supports aliased keys in the index. Removed the uniqueness parsing for the last section of the key name. ES v2.x and on requires the keys be fully qualified. * es-search.pl subaggs output adjusted Prior version injected the sub aggregations before the values, this now append the sub aggregations to the listed output in a more readable manner. * Allow aggregations to be wrapped and use it to provide date_histogram wrappers for es-search.pl es-search.pl now supports --interval to wrap aggregations in a date_histogram's so you can better understand what is happening with your data. * es_master() and --master-only added, documentation clean up. ================================================= Plus 59 releases after 2017-01-02 00:00:00 +0000. =================================================