Changelog

Info

This page mirrors the project CHANGELOG.md. See the Releases page for downloads.

2.0.0 [2026/07/31] - Black Hat Arsenal USA 2026 Release

New Features:

• Added the azure-timeline command to create a DFIR timeline for Azure logs. (#109) (@fukusuket)
• New aws-ct-search command to search through CloudTrail logs. (#117) (@fukusuket)
• Added support for an ignore-list file (config/aws_ignore_rule_list.txt) to skip loading rules by UUID, so superseded/duplicate rules can stay in the repo without being loaded. (#136) (@YamatoSecurity)
• Added Parquet (.parquet) input support to the AWS commands (aws-ct-timeline, aws-ct-metrics, aws-ct-summary, aws-ct-search) for both -f and -d. Each row is converted to one JSON event: nested struct columns become nested objects, the known CloudTrail envelope fields (userIdentity, requestParameters, responseElements, …) stored as serialized JSON strings — the shape Athena CTAS / Glue / Firehose pipelines produce — are parsed back into objects so rules can match nested values, and timezone-less eventTime TIMESTAMP columns are marked as UTC so time filtering and summaries keep working. Snappy, gzip, zstd, and lz4 compression are supported. (@nishikawaakira)

Enhancements:

• Replaced the wildcard ("*") version requirements on ten dependencies with explicit floors, so a future breaking release is a deliberate bump rather than something a cargo update accepts silently. Six of them are pre-1.0, where every minor is a breaking change under Cargo's semver rules. This is the failure mode behind #131, where a wildcard-pinned cidr-utils shipped a backend that stopped accepting abbreviated CIDR and --geo-ip began panicking at startup with nothing in Suzaku having changed. No resolved version changes: cargo update produces an identical Cargo.lock with and without the floors. (#189) (@YamatoSecurity)
• Errors that end the run are now all printed in red and all honor --no-color. Previously the color was inconsistent: some (a nonexistent input path, an invalid -m, --min-level, a missing rules folder or output profile, an existing output file without -C, a failed update-rules) were printed in the default terminal color, and a few others were hard-coded red that ignored --no-color. They now share a single error_msg() helper with fatal_error(). (@fukusuket)
• Warnings and errors raised while processing logs (skipped files, a failed directory scan, correlation-event errors, a missing abused-AWS-API list) are now written to logs/errorlog-<YYYYMMDD_HHMMSS>.log instead of the terminal, so they no longer interleave with the progress bar or bury the results summary. The file is created only when there is something to report, its first line is the command line that produced it, and the run ends with a single Warnings and errors: <n> messages saved to <path> pointer. Fatal errors that abort the run are still printed to the terminal as well. (@fukusuket)
• Reworked -t, --output-type for aws-ct-summary to take format names like the timeline commands, and added DuckDB output. Pass csv, json, jsonl, and/or duckdb (comma-separated or repeated), e.g. -t csv,duckdb. Because the CSV folds each principal's API calls and attributes into multi-line cells that cannot be queried, the DuckDB output stores them relationally across three tables — summary (one row per principal), summary_api_calls (one row per principal/API, labelled abused_success/abused_failed/other_success/other_failed) and summary_attributes (one row per principal/value, labelled aws_region/src_ip/access_key_id/user_agent) — so questions the CSV cannot answer, such as which source IPs were used by the principals that called an abused API, become ordinary joins. (Breaking: the numeric -t 1..5 form is replaced by names, matching the other commands.) (@YamatoSecurity)
• Reworked -t, --output-type for aws-ct-timeline and azure-timeline to take format names instead of numbers, and added DuckDB output. Pass csv, json, jsonl, and/or duckdb (comma-separated or repeated) to write any combination at once, e.g. -t csv,duckdb; DuckDB output is a self-contained .duckdb database with a timeline table whose columns are the output-profile fields. (Breaking: the numeric -t 1..5 form is replaced by names; aws-ct-search shares the option and gains the same named formats.) (@YamatoSecurity)
• Added a Tags column to the aws-ct-timeline and azure-timeline output that renders a rule's Sigma tags list as a single ¦-separated string (like Hayabusa) instead of dropping the list. ATT&CK tactics are abbreviated using the editable config/mitre_tactics.txt table shared with Hayabusa (e.g. attack.credential-access becomes CredAccess), while techniques and groups are shortened (attack.t1562.001 becomes T1562.001, attack.g0035 becomes G0035); both the hyphen and underscore tactic spellings are handled. JSON output keeps the value as a flat string. (#62) (@YamatoSecurity)
• Added a -l, --localtime option to the aws-ct-timeline and azure-timeline commands that outputs event timestamps in the machine's local timezone with an explicit UTC offset (e.g. 2023-07-10 12:27:45 becomes 2023-07-10 21:27:45+09:00 in JST) instead of UTC. Unparseable timestamps fall back to the existing UTC rendering. (#34) (@YamatoSecurity)
• Bumped sigma-rust to the released v0.7.1 and updated all other dependencies to their latest versions. sigma-rust v0.7.1 keeps the Sigma correlation support suzaku relies on while moving its YAML backend off the deprecated serde_yml/noyalib (which parsed large unsigned 64-bit values in rules/events as lossy floats) to the actively maintained yaml_serde, restoring correct u64 parsing. (@YamatoSecurity)
• Code refactored for easier handling of different log sources. (@fukusuket)
• Added support for Microsoft Graph API JSON format for Azure logs. (#113) (@fukusuket)
• azure-timeline now unwraps the { "records": [...] } batch envelope used by Azure Monitor diagnostic-settings blobs and Event Hub messages (both whole-file and per-line), so those exports are read record-by-record instead of as a single event, and it now loads/matches the identity_protection (riskdetection) and privileged_identity_management (pim) rule types, which were previously dropped at load. (#130) (@YamatoSecurity)
• azure-timeline now loads and matches SigmaHQ's Microsoft 365 rules, which declare logsource.service as audit/exchange/threat_detection/threat_management — only m365 was recognized before, so every upstream m365 rule was dropped at load. These services are routed through the same Workload/RecordType Unified Audit Log discriminator. (#137) (@YamatoSecurity)
• Added support for the M365 Unified Audit Log to azure-timeline: reads Search-UnifiedAuditLog CSV exports (and JSON) by unwrapping the AuditData column/wrapper, folds UAL Name/Value property bags (ExtendedProperties/Parameters/…) into objects so rules can match nested values (e.g. ExtendedProperties.UserAgent), parses single/pretty-printed record objects, no longer drops events when no time filter is set, parses the CreationTime timestamp, and adds an m365 log-source service. The Azure output profile now surfaces DFIR-relevant M365 fields (Workload, Operation, Result, User, SrcIP, TargetObject, UserAgent, AppId, LogonError, and a Details summary of the change's Parameters/ModifiedProperties) instead of the previously empty Azure-Monitor-only columns. (#129) (@YamatoSecurity)
• Added --file-date-from/--file-date-to options that filter objects by their S3 key date prefix, distinct from the existing --timeline-start/--timeline-end options, which operates on in-file event timestamps. (#118) (@fukusuket)
• Added -output-type option for the aws-ct-summary command to output in JSON. (#123) (@fukusuket)
• aws-ct-metrics can now aggregate several fields in a single scan and enrich the results. -F, --field-name takes a comma-separated list (e.g. -F sourceIPAddress,userAgent,userIdentity.arn,awsRegion,userIdentity.accessKeyId) and tallies every field in one pass over the logs, instead of requiring a full scan per field; each value additionally reports the FirstSeen/LastSeen of that value; -G, --geo-ip adds SrcASN/SrcCity/SrcCountry columns for values that parse as IP addresses; -t, --output-type writes csv/json/jsonl/duckdb (the DuckDB output is a single queryable metrics table); and -s, --include-sts-keys matches aws-ct-summary by excluding temporary ASIA... STS keys unless asked for. Events that do not have the requested field are now counted as - rather than dropped, so the percentages are a share of every event scanned. -F is also validated against the CloudTrail record fields before the scan starts, so a mis-cased name fails immediately with 'sourceIPaddress' is not a CloudTrail field. Did you mean 'sourceIPAddress'? instead of scanning the whole dataset to report - for every event; any sub-path of an API-specific container (requestParameters, responseElements, additionalEventData, ...) is still accepted as-is. (Breaking: the CSV columns changed from EventName,Percent,Total to Field,Value,Count,Percent,FirstSeen,LastSeen.) (@fukusuket)

Bug Fixes:

• --geo-ip reported every routable IPv6 address as Private and never looked it up: 2000::/3, the entire IPv6 global unicast space, was listed as a private range. Public addresses such as a Japanese consumer ISP or an Azure sign-in from anywhere rendered SrcASN: Private / SrcCountry: -, which does not just omit the geolocation but asserts something untrue — an analyst reading Private concludes the traffic was internal. IPv6 is now geolocated like IPv4; unspecified, link-local, unique-local and multicast addresses keep their placeholders. (#182) (@YamatoSecurity)
• --geo-ip left the geolocation columns empty whenever an earlier field in the profile's SrcIP fallback list was present but unusable. The lookup took the first candidate that existed and only then tried to parse it, so a claims.ipaddr holding "", "-", a host:port pair (the common M365 ClientIP shape) or any other non-IP ended the search and the routable address sitting in callerIpAddress was never consulted — SrcASN/SrcCity/SrcCountry rendered - for a record that plainly carried a public IP. The search now takes the first candidate that parses as an IP address, and the SrcIP column resolves through the same selector, so the address displayed and the address geolocated are always the same field. When no candidate parses, SrcIP still shows what the log recorded (an AWS-service event writes cloudtrail.amazonaws.com there) and the geo columns show -. (#183) (@YamatoSecurity)
• The aws-ct-metrics table and CSV always labelled the first column EventName, even when -F aggregated a different field, so a run over sourceIPAddress produced a column of IP addresses headed EventName. The column is now named after the field that was actually aggregated. The command also silently overwrote an existing -o file; overwriting now requires -C, --clobber, as it does for every other command. Values with equal counts are additionally sorted by value, so repeated runs produce byte-identical output instead of shuffling rows in HashMap order. (@fukusuket)
• The Results Summary "Data reduction" line panicked (attempt to subtract with overflow) in debug builds — or printed a nonsensical ~1.8×10¹⁹ count in release, or NaN% on empty input — because event_with_hits (which correlation results increment for events already counted by the base scan) could exceed total_events. The count now saturates and the percentage is guarded against an empty dataset. (#163) (@YamatoSecurity)
• aws-ct-summary reported the wrong per-entry time range: the first_seen/last_seen of every summarized region, source IP, access key, user agent, and API were seeded once from the dataset-global min/max at the moment the key was first inserted and never updated, so they showed the whole dataset's span rather than that entry's own first/last occurrence. Each entry now tracks the first/last event time of the events that actually hit it. (#160) (@YamatoSecurity)
• Scanning now warns ([WARNING] Skipping <file>: <reason>) when it skips an input file it could not read — permission denied, non-UTF-8 content, a file removed mid-scan, or a corrupt/oversized .gz — instead of silently dropping it. Previously such a file was counted in the total but skipped with no indication, so the reported coverage was overstated. Applies to both the directory scan and single-file input, and the gzip size-cap warning was moved to this single call-site path. (#161) (@YamatoSecurity)
• aws-ct-search no longer panics on an invalid --regex value (it was compiled with .expect()); an invalid pattern now prints a clear error and exits cleanly. A malformed --filter missing the FIELD:VALUE colon — previously silently ignored — is now rejected up front. And aws-ct-summary's warning when the abused-AWS-API list can't be opened now names the path it looked for and states that all API calls will be classified as non-abused. (#162) (@YamatoSecurity)
• --geo-ip enrichment (SrcASN/SrcCity/SrcCountry) was a silent no-op for azure-timeline/M365 logs: the source IP was resolved from a hardcoded sourceIPAddress field, which only exists in AWS CloudTrail. The Azure/M365 profile maps SrcIP to callerIpAddress/ClientIP/etc., so those columns always rendered - even for events with a routable public IP. The lookup now resolves the source IP from the profile's SrcIP field spec (the same |-separated fallback used for other columns), so Azure/M365 enrich just like AWS. (#159) (@YamatoSecurity)
• Malformed --timeline-start / --timeline-end / --time-offset values are now rejected up front with a clear error, instead of being parsed per-event and silently dropping every event (empty timeline, no warning) — e.g. a plain --timeline-start 2024-01-01 instead of full RFC 3339. Also fixed a parse_offset panic on an empty offset, trailing whitespace, or a multibyte trailing character (the split index was taken from the untrimmed length). (#150) (@YamatoSecurity)
• Fixed a panic (byte index 24 is not a char boundary) when the end-of-scan "Rule Authors" summary truncated an author name longer than 27 bytes whose 24th byte fell mid-codepoint — routine for Japanese/CJK and other non-ASCII author names common in Sigma rule packs. Truncation now counts and cuts by characters, not bytes, so the completed run's output is no longer discarded. (#148) (@YamatoSecurity)
• Neutralized CSV/spreadsheet formula injection (CWE-1236) in report output. CSV cells come from attacker-influenceable cloud-log fields (userAgent, principal ARNs, error strings, …); a value beginning with =, +, -, @, tab, or CR would be evaluated as a formula when the report is opened in Excel/LibreOffice/Sheets. Such values are now prefixed with an apostrophe (spreadsheets treat it as a force-text marker) at all CSV sinks; JSON/JSONL and stdout are unchanged. (#146) (@YamatoSecurity)
• Bounded gzip decompression to stop a decompression-bomb OOM: a crafted or corrupt .gz file anywhere in the scanned tree could inflate to many GB (DEFLATE reaches ~1032:1) and get the entire scan OOM-killed. .gz inputs are now capped at 3 GiB decompressed, and an over-limit file is skipped with a warning instead of aborting the run. (#147) (@YamatoSecurity)
• --geo-ip corrupted the timeline output: when a record's sourceIPAddress was not a parseable IP address (routine for AWS-service events such as cloudtrail.amazonaws.com), the GeoIP lookup returned that raw string for every output column, overwriting Timestamp, EventName, RuleTitle, etc. Enrichment is now scoped to the SrcASN/SrcCity/SrcCountry columns only, which fall back to - when the address cannot be resolved. (#145) (@YamatoSecurity)
• Hardened the input/filesystem edge cases that previously aborted a scan with a Rust panic and backtrace: a non-UTF-8 filename anywhere in the scanned tree (which killed the initial file count before any results), an unreadable subdirectory or a file removed mid-scan, and an unwritable --output path now produce a clean, actionable error instead — the walk errors are reported and the output error exits non-zero with a Cannot write to output file … message. Applies across aws-ct-timeline, azure-timeline, aws-ct-search, aws-ct-metrics, and aws-ct-summary. Non-UTF-8 filenames are now read (the real path is kept through the scan pipeline) instead of being counted but skipped, and aws-ct-summary's JSON/JSONL output also reports an unwritable --output path cleanly. (#149) (@YamatoSecurity)
• The aws-ct-timeline, aws-ct-metrics, aws-ct-search, and aws-ct-summary commands silently dropped JSONL input (one CloudTrail event, or a { "Records": [...] } batch, per line): the parsers read the whole file as a single JSON document and returned no events when that failed. They now fall back to per-line JSONL parsing, and .jsonl files are discovered and read. (#139) (@YamatoSecurity)
• -T, --no-frequency-timeline option was not working so we removed it. Also fixed a logic bug in the authors display. (#110) (@fukusuket)
• Output file would get saved even if there were no results. (#114) (@fukusuket)
• aws-ct-summary would panic when processing a corrupt or imcomplete log file. (#119) (@fukusuket)
• --geo-ip would panic at startup (invalid IP address syntax) because the abbreviated CIDR strings used for the private-IP check (e.g. 10/8, 172.16/12, 2000::/3) are no longer accepted by the cidr crate. Dropped the cidr-utils dependency and check private ranges directly with std's Ipv4Addr::is_private() and a manual IPv6 prefix match; also populated the previously unused GeoIP country/city caches. (#132) (@fukusuket)

1.1.0 [2025/08/14] - Obon Release

Enhancements:

• -R, --raw-output now outputs raw logs to the terminal when -o is not specified. (#101) (@fukusuket)

1.0.1 [2025/08/07] - Black Hat Arsenal USA 2025 Release

Bug Fixes:

• Better error handling for invalid file and directory input. (#99) (@fukusuket)

1.0.0 [2025/07/31] - Black Hat Arsenal USA 2025 Release

New Features:

• Added support for correlation rules (event_count, value_count, temporal, temporal_order) for the aws-ct-timeline command. (#97) (@fukusuket)

Enhancements:

• Level names are now abbreviated in aws-ct-timeline. (#68) (@fukusuket)
• Error message output when no rules are found. (#76) (@fukusuket)
• Added --timeline-offset, --timeline-start and --timeline-end options to the aws-ct-timeline command. (#58) (@fukusuket)
• aws-ct-timeline now runs with multi-threading. (#32, #93) (@hach1yon)

0.2.1 [2025/05/25] - AUSCERT/SINCON Release 2

• Fixed the release name and updated the readme. (@yamatosecurity)

0.2.0 [2025/05/22] - AUSCERT/SINCON Release

New Features:

• aws-ct-summary: for each unique ARN, creates a summary of total events, regions used, user types, access keys, user agents, etc... (#53) (@fukusuket)

Enhancements:

• Added Maxmind geolocation information to source IP addresses for the aws-ct-timeline and aws-ct-summary commands. (#16) (@fukusuket)
• Added a -R, --raw-output option to the aws-ct-timeline command to output the original JSON data when there is a detection. (#67) (@fukusuket)

Bug Fixes:

• The CSV headers for the aws-ct-metrics command were incorrect. (#72) (@fukusuket)

0.1.1 [2025/04/24] - AlphaOne Release

Bug Fixes:

• Some Sigma fields were not being outputted properly. (#61) (@fukusuket)

Initial Release

0.1.0 [2025/04/20] - AlphaOne Release

New Features:

• aws-ct-metrics: create metrics for AWS CloudTrail events
• aws-ct-timeline: perform sigma detection on AWS CloudTrail logs
• update-rules: update sigma rules