1.063 2023-07-08 Released-By: PERLANCAR; Urgency: low - No functional changes. - Remove use of smartmatch. 1.062 2022-11-29 Released-By: PERLANCAR; Urgency: medium - [ux] Add short option alias -B for --render-as-benchmark-pm. 1.061 2022-02-08 Released-By: PERLANCAR; Urgency: medium - [bugfix] Fix missing schema in format_result()'s Rinci metadata. 1.060 2021-08-13 Released-By: PERLANCAR; Urgency: high - Merge releases 1.059 and 1.059.2. 1.059.2 2021-08-13 Released-By: PERLANCAR; Urgency: medium - [ux] Add option --render-as-benchmark-pm. - UPDATE 2021-08-13: rename release from 1.056 to 1.059.2 due to duplicate. 1.059 2021-07-31 Released-By: PERLANCAR; Urgency: medium - [bugfix] Fix goto bug that caused empty item names. - [action list-items] Show _succinct_name for debugging. 1.058 2021-07-31 Released-By: PERLANCAR; Urgency: medium - [bugfix] When rendering result as Benchmark.pm result, fix calculating rate from time (time is in ms not s). - [bugfix] Avoid mangling non-function participant names when creating succinct names (e.g. 'perl -e (baseline)'. 1.057 2021-07-23 Released-By: PERLANCAR; Urgency: high - Doh, typo. 1.056 2021-07-23 Released-By: PERLANCAR; Urgency: medium - chart_result(): Change to use items directly, and use _succinct_name. - Tweak/fix logging statement. 1.055 2021-07-23 Released-By: PERLANCAR; Urgency: medium - Add new renderer formatter: RenderAsBenchmarkPm. 1.054 2021-07-23 Released-By: PERLANCAR; Urgency: medium - Split Sah schemas to its own dist Sah-Schemas-Bencher. 1.053 2021-04-10 Released-By: PERLANCAR; Urgency: medium - Update dependency's new name {,Data::}TableData::Object. 1.052 2020-09-21 Released-By: PERLANCAR; Urgency: medium - Improve multimodver/multiperl mode: Do not dump _code (coderef), but reconstruct from _code_str instead. - Allow participant that does not mention module to be permuted in multimodver mode. 1.051 2020-08-16 Released-By: PERLANCAR; Urgency: high - Merge two releases 1.050 and 1.050.1. 1.050.1 2020-08-15 Released-By: PERLANCAR; Urgency: medium; Status: Cancelled - Add bencher options: tidy, keep_tempdir. - [Rinci metadata] Tweak argument categories. - UPDATE 2020-08-16: rename release from 1.050 to 1.050.1 and cancel (duplicate). 1.050 2020-06-18 Released-By: PERLANCAR; Urgency: medium - Observe module_startup_precision property. 1.049 2020-05-10 Released-By: PERLANCAR; Urgency: medium - When profiling, observe --precision to set a repetition loop. 1.048 2020-04-29 Released-By: PERLANCAR; Urgency: low - Tweak Rinci metadata: replace deprecated x.schema.{,element_}entity with more specific schema. - Avoid perl undef warning. 1.047 2020-01-12 Released-By: PERLANCAR; Urgency: medium - Allow specifying field format in add_field(). - [experimental] Replace formatter AddVsSlowest with AddComparisonFields (which adds pct_faster_vs_slowest, pct_slower_vs_fastest) to make it easy to say "xx times/percent slower than ..." as well as "xx times/percent faster than ...". 1.046 2019-11-29 Released-By: PERLANCAR; Urgency: high - Merge releases 1.045 & 1.045.1. 1.045.1 2019-11-29 Released-By: PERLANCAR; Urgency: medium - Update coerce rule names (Data::Sah::Coerce 0.034+). UPDATE 2019-12-02: rename release from 1.045 to 1.045.1, cancel release (duplicate version). 1.045 2019-10-15 Released-By: PERLANCAR; Urgency: medium - Update renamed prereq Data::Clean::JSON -> Data::Clean::ForJSON. 1.044 2019-02-24 Released-By: PERLANCAR - Support loading scenario from an Acme::CPANModules module (add --cpanmodules-module option). 1.043 2019-02-24 Released-By: PERLANCAR - Use Exporter to check imported args. 1.042 2019-02-15 Released-By: PERLANCAR - [ux] Allow specifying scenario module with Bencher::Scenario:: prefix. - Stringify arg_XXX result column. 1.041 2018-04-03 Released-By: PERLANCAR - Implement import_args & import_args_array. - [ux] In multimodver mode, allow if there is only one path/version of the module. - Do not turn on --with-process-size on --module-startup or --code-startup because they are meaningless, the participant is of type=command. 1.040 2017-07-13 Released-By: PERLANCAR - Add code_startup mode. 1.039 2017-07-02 Released-By: PERLANCAR - [Bugfix] Incomplete conversion to Log::ger. 1.038 2017-07-02 Released-By: PERLANCAR - [ux] If there is only one item and we are in interactive mode, open the profile result in browser. 1.037 2017-07-02 Released-By: PERLANCAR - Add actions: dump-parsed-scenario, dump-items, profile (to profile items with Devel::NYTProf). - Replace Log::Any with Log::ger. 1.036 2017-02-20 Released-By: PERLANCAR - [Bugfix] Fix completion of participant & dataset tags. 1.035 2017-02-19 Released-By: PERLANCAR - Show participant tags (p_tags) & dataset tags (ds_tags) columns in result. 1.034 2017-01-25 Released-By: PERLANCAR - Introduce participant property: helper_modules (allow specifying modules that are require for running code but are not subject of benchmark). - Return Bencher::Backend version in result meta. - [ux] Show elapsed time when output format is text. 1.033 2017-01-04 Released-By: PERLANCAR - [ux] Format numbers etc when output format is /html/, show platform information when output format is text. 1.032 2016-12-10 Released-By: PERLANCAR - Turn on with_process_size by default when in module_startup mode on linux. 1.031 2016-11-23 Released-By: PERLANCAR - Record scenario file's mtime and checksums (MD5, SHA1, SHA256) in result metadata. 1.030 2016-10-28 Released-By: PERLANCAR - [ux] Provide completion for --sort/--sorts. 1.029 2016-10-27 Released-By: PERLANCAR - Add argument: scenario (to pass scenario data structure). 1.028 2016-10-20 Released-By: PERLANCAR - Do not require Sys::Load on Windows because it doesn't install there. Bencher now works on Windows. 1.027 2016-10-14 Released-By: PERLANCAR - Add options: --exclude-xs-modules (--noxs) --exclude-pp-modules (--nopp). 1.026 2016-10-13 Released-By: PERLANCAR - Add hooks before_parse_{datasets,participants} and deprecated before_list_{datasets,participants}. To generate datasets/participants dynamically, use the before_parse_* hooks because the datasets/participants can later still be filtered. - Add hooks 'before_test_item' & 'after_test_item'. 1.025 2016-10-09 Released-By: PERLANCAR - [Bugfix] Several bug fixes. - Add property/option 'test' to skip testing participant code prior to benchmarking. Useful if code runs for at least a few seconds and we don't want to wait for too long. 1.024 2016-10-09 Released-By: PERLANCAR - [Incompatible change] Remove action 'bench-with-benchmark-pm' (see below to run benchmark with Benchmark.pm). - Add property/option 'runner' which can be the default 'Benchmark::Dumb', or 'Benchmark', or the new 'Benchmark::Dumb::SimpleTime' which can be used if you want to benchmark code that runs for at least a few seconds with very few iterations (like 1 or 2) without "Number of runs too small" warning from Benchmark::Dumb. 1.023 2016-09-09 Released-By: PERLANCAR - Aside from observing per-dataset 'result' property, also observe scenario's 'result' property for when we don't have any datasets. 1.022 2016-09-09 Released-By: PERLANCAR - [workaround] Some completions were broken because we use 'str_comma_sep' coercion rules for --include-path, this is because include_path is now a string instead or array[str] before validation+coercion, and completion currently does not use Data::Sah, so now we disable this coercion rule for now. 1.021 2016-08-26 Released-By: PERLANCAR - Introduce --with-args-sizes. 1.020 2016-08-24 Released-By: PERLANCAR - [Incompatible change] Rename options for consistency --include-process-size to --with-process-size, --include-result-size to --with-result-size (--include-* are for filter options). - [ux] Add completion for comma-separated filter options, e.g. --include-participants, --exclude-dataset-seqs, etc. 1.019 2016-08-24 Released-By: PERLANCAR - [ux] Allow specifying options like --include-datasets with comma-separated value, so --include-datasets 1,2,3 instead of --include-dataset 1 --include-dataset 2 --include-dataset 3 or --include-datasets-json '[1,2,3]'. - [Bugfix][Regression] Accidentally removed a code to permute dataset, breaking v1.018. 1.018 2016-08-20 Released-By: PERLANCAR - UPDATE: this release is broken, don't use. [NEW FEATURES] - Allow items to be benchmarked using Benchmark.pm instead of Benchmark::Dumb. To do this, specify bencher option --bench-with-benchmark-pm. Currently if you do this, the raw result from Benchmark.pm will be used and no result table will be returned/saved. Also don't forget to set --benchmark-pm-count when appropriate (the default is -0.5). - Introduce scenario property 'env_hashes' (can also be set from the CLI using --env-hash). This allows permuting multiple environment variable sets. 1.017 2016-07-13 Released-By: PERLANCAR - Add chart_result(). 1.016 2016-07-13 Released-By: PERLANCAR - Include permutation in result metadata (func.permute). 1.015 2016-07-12 Released-By: PERLANCAR - Handle negative time in results. - [Bugfix] Still allow stringy matching of --include with name even though value is a number. 1.014 2016-06-30 Released-By: PERLANCAR - Add function: split_result(). 1.013 2016-06-29 Released-By: PERLANCAR - [Bugfix] Determine number of significant digits from 'time', not from 'rate' because 'rate' is removed in module startup result. 1.012 2016-06-28 Released-By: PERLANCAR - format_result(): Add a convenient way to skip using the RenderAsTextTable formatter by passing option render_as_text_table=>0. - Avoid some perl undef warnings, e.g. for older bencher results where 'perl' field is not defined. 1.011 2016-06-23 Released-By: PERLANCAR - More tweaks/fixes for alignments. 1.010 2016-06-17 Released-By: PERLANCAR - [Bugfix] DeleteConstantFields formatter didn't use delete_fields() yet. 1.009 2016-06-17 Released-By: PERLANCAR - [ux] Add column alignment to result. - [ux] Use decimal notation by default, except for 'errors' field or when option --scientific-notation is given. 1.008 2016-05-20 Released-By: PERLANCAR - Follow Sah::Schemas specification, rename Sah::Schema::Bencher::Scenario to Sah::Schema::bencher::scenario. - Avoid perl undef warning. 1.007 2016-04-19 Released-By: PERLANCAR - [Bugfix] save_result was turned *on* all the time, it should be only when --save-result or (--result-dir or BENCHER_RESULT_DIR was set). 1.006 2016-04-13 Released-By: PERLANCAR - Add --{in,ex}clude-{participant,dataset,item}-{name,seq} to unambiguously include/exclude by name or sequence. - [ux] Warn when there is a participant/dataset/item which has name that is also a sequence number, which can be confusing when including/excluding. - [Bugfix] Defeat the annoying compile errors when using Capture::Tiny due to mismatching prototypes. 1.005 2016-04-08 Released-By: PERLANCAR [ENHANCEMENTS] - Add --include-process-size to show process memory stats. UPDATE: this option has been renamed to --with-process-size in 1.020. - Add output capturing (--show-items-outputs, --capture-stdout, --capture-stderr). [BUG FIXES] - Regression: --show-items-results and --show-items-results-sizes returned JSON by default when it should return a formatted text. - When trying to come up with a name for a command participant, also take a look at cmdline_template/perl_cmdline/perl_cmdline_template in addition to just cmdline. 1.004 2016-04-08 Released-By: PERLANCAR - Eval code template in 'main' package. - Allow in template to avoid escaping. 1.003 2016-04-03 Released-By: PERLANCAR - [Bugfix] completer for participant/dataset name didn't include generated names. - [Bugfix] save_result was not turned on automatically when user sets result_dir. 1.002 2016-04-01 Released-By: PERLANCAR - [Bug] Logic bug: --return-meta accidentally defaulted to off when --save-result is not enabled. 1.001 2016-04-01 Released-By: PERLANCAR - Turn off include_result_size when in module startup mode. - Add feature to save result to a JSON file (--save-result, --result-dir, --result-filename). - Grok scenario property 'extra_modules' to allow extra modules to be loaded and their versions recorded in benchmark result metadata. 1.000 2016-03-31 Released-By: PERLANCAR - No functional changes. - Bump version number to 1.000. 0.46 2016-03-31 Released-By: PERLANCAR - No functional changes. - [doc] Add description, fixes. 0.45 2016-03-30 Released-By: PERLANCAR - [ux] Check that what's specified in --{in,ex}clude-{dataset,participant,item} are actually known, to avoid silent surprise. - [Bugfix] Fix typo that caused sometimes specifying --include-participant by name doesn't work. - [Bugfix] RoundNumbers formatter: no need to round result_size except when it's not a round number. 0.44 2016-03-26 Released-By: PERLANCAR - Show result size (as measured by Devel::Size), add --include-result-size (& include_result_size scenario property), add action show-items-results-sizes. UPDATE: the option has been renamed to --with-result-size in 1.020. - [Bugfix] Fix completion of -m when there is -I specified. 0.43 2016-03-24 Released-By: PERLANCAR - [ux] Directory specified in -I (--include-path) is now also used for searching and loading scenario module, this is so we can say something like 'bencher -Ilib -m Foo' instead of 'PERL5OPT=-Ilib bencher -m Foo'. - [ux] When a dataset has no name, try to assign a name from argument value. 0.42 2016-03-22 Released-By: PERLANCAR - Allow specifying 'modules' in participant to benchmark startup overhead of several modules at once. 0.41 2016-03-18 Released-By: PERLANCAR - No functional changes. - [Bugfix] Include modver & perl in result items even when in not multiperl/multimodver mode (ModuleStartup formatter module uses the perl key). - Add workaround for RT#113117 (List::MoreUtils sometimes return undef max). 0.40 2016-03-16 Released-By: PERLANCAR - Add action: list-perls; by default only use perls that have Bencher::Backend installed. - [Bugfix] Avoid benchmarking duplicate items, e.g. 'perl -e1 (baseline)' when in module_startup mode + multimodver mode. - [Bugfix] fix calculating mod_overhead_time when in multiperl mode. 0.39 2016-03-15 Released-By: PERLANCAR [ENHANCEMENTS] - Support benchmarking against multiple perls (introduce options `--multiperl`, `--include-perl`, `--exclude-perl`). Limitation: Currently this is implemented by dumping (using Data::Dmp, which means using B::Deparse) the items' codes into a temporary scenario file, then executing the scenario file using each perl executable. This means codes that contains closures Released-By: references to outside variable won't work correctly. - Support benchmarking multiple versions of a module (`--multimodver`). To do this, first add more module search paths using `--include-path` (`-I`). - In participant, introduce 'perl_cmdline' and 'perl_cmdline_template' in addition to 'cmdline' and 'cmdline_template'. They are the same as 'cmdline*' except there is implicit first argument which is the perl executable. They are for executing commands using perl, without having to hardcode/include the perl executable location. 0.38 2016-03-15 Released-By: PERLANCAR - First release, split from Bencher distribution to reduce dependencies when installing Bencher to a perl installation (this is part of developing support for benchmarking against multiple perls/module versions).