[Recorded Future] Add support for code repo leakage event - #20847
[Recorded Future] Add support for code repo leakage event#20847moxarth-rathod wants to merge 1 commit into
Conversation
moxarth-rathod
added
enhancement
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
✅ Elastic Docs Style Checker (Vale)No issues found on modified lines! The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale. |
|
Changelog link mismatch — expected
Tip If expected, add the |
💔 Build Failed
Failed CI StepsHistory |
![vera-review-bot[bot]](https://avatars.githubusercontent.com/in/3297022?s=60&v=4){kind=small}
| - set: | ||
| field: rule.name | ||
| tag: set_rule_name_from_playbook_alert_panel_status_alert_rule_name | ||
| copy_from: recordedfuture.playbook_alert.panel_status.alert_rule.name |
There was a problem hiding this comment.
Severity: 🟡 Medium confidence: medium path: packages/ti_recordedfuture/data_stream/playbook_alert/elasticsearch/ingest_pipeline/default.yml:529
rule.name is copied from panel_status.alert_rule.name but that source field is never added to the duplicate-field cleanup list, so the value is stored twice; add it to the remove_custom_duplicate_fields_d7f69209 processor.
Details
This pipeline's convention is that every custom field copied into an ECS field is dropped by the remove_custom_duplicate_fields_d7f69209 processor (line 661) unless the preserve_duplicate_custom_fields tag is set -- panel_status.case_rule_id (-> rule.id), compromised_host.computer_name (-> host.name), malware_family.name (-> threat.indicator.name) are all listed there. The new rule.name copy source, recordedfuture.playbook_alert.panel_status.alert_rule.name, is not, so it stays in the document alongside rule.name for every code_repo_leakage alert. The pipeline test does not catch this because _dev/test/pipeline/test-common-config.yml sets the preserve_duplicate_custom_fields tag for all fixtures, so the removal branch is never exercised.
Recommendation:
Add the copied source field to the duplicate-removal list:
- remove:
field:
- recordedfuture.playbook_alert.panel_evidence_summary.compromised_host.computer_name
- recordedfuture.playbook_alert.panel_evidence_summary.compromised_host.os
- recordedfuture.playbook_alert.panel_evidence_summary.malware_family.id
- recordedfuture.playbook_alert.panel_evidence_summary.malware_family.name
- recordedfuture.playbook_alert.panel_status.alert_rule.name
- recordedfuture.playbook_alert.panel_status.case_rule_id
- recordedfuture.playbook_alert.panel_status.created
- recordedfuture.playbook_alert.panel_status.creator_id
- recordedfuture.playbook_alert.panel_status.creator_name
- recordedfuture.playbook_alert.panel_status.risk_score
- recordedfuture.playbook_alert.panel_status.updated
- recordedfuture.playbook_alert.playbook_alert_id
tag: remove_custom_duplicate_fields_d7f69209
ignore_missing: true
if: ctx.tags == null || !(ctx.tags.contains('preserve_duplicate_custom_fields'))🤖 AI-Generated Review | Vera Review Bot - v0.2.6 | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| formats: | ||
| - ISO8601 | ||
| on_failure: | ||
| - remove: |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: high path: packages/ti_recordedfuture/data_stream/playbook_alert/elasticsearch/ingest_pipeline/default.yml:265
Two of the newly added processors (the on_failure remove at line 265 and the nested foreach at line 273) have no tag, unlike every other processor in this pipeline; add tags to both.
Details
Version 2.8.0 of this package was released specifically to add tags to every ingest pipeline processor, and the immediately preceding, structurally identical processors do carry them (remove_ingest_value_published_056ab9b2 at line 250, foreach_ingest_value_changes_ecc8b712 at line 491). The two new untagged processors break that convention, which makes failures harder to attribute because the pipeline-level on_failure message interpolates _ingest.on_failure_processor_tag. Tags become enforced by elastic-package check at format_version >= 3.6.0; this package is on 3.3.2, so this is a consistency issue rather than a build failure.
Recommendation:
Add tags to both new processors:
- foreach:
field: recordedfuture.playbook_alert.panel_evidence_summary.evidence
if: ctx.recordedfuture?.playbook_alert?.panel_evidence_summary?.evidence instanceof List
tag: foreach_panel_evidence_summary_evidence
processor:
date:
field: _ingest._value.published
tag: date_panel_evidence_summary_evidence_published
target_field: _ingest._value.published
formats:
- ISO8601
on_failure:
- remove:
tag: remove_ingest_value_evidence_published
field: _ingest._value.published
ignore_missing: true
- foreach:
field: recordedfuture.playbook_alert.panel_evidence_summary.evidence
if: ctx.recordedfuture?.playbook_alert?.panel_evidence_summary?.evidence instanceof List
tag: foreach_panel_evidence_summary_evidence_targets
processor:
foreach:
tag: foreach_ingest_value_evidence_targets
field: _ingest._value.targets
ignore_failure: true
processor:
append:
field: related.hosts
tag: append_evidence_target_name_into_related_hosts
value: '{{{_ingest._value.name}}}'
allow_duplicates: false🤖 AI-Generated Review | Vera Review Bot - v0.2.6 | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| # newer versions go on top | ||
| - version: "2.9.0" | ||
| changes: | ||
| - description: Add support for code repo leakage event in playbook alert data stream. |
There was a problem hiding this comment.
Severity: 🔵 Low confidence: medium path: packages/ti_recordedfuture/changelog.yml:4
The 2.9.0 changelog covers only the new event type, but the PR also fixes event.severity for the Moderate priority across all playbook alert categories; add a bugfix entry for it.
Details
The severity script at default.yml line 595 now also matches moderate. Recorded Future's priority enum uses Moderate (not Medium), so before this change every playbook alert of any category with priority Moderate fell through all branches and got no event.severity at all. That is a user-visible behaviour change for the pre-existing domain_abuse, cyber_vulnerability and identity_novel_exposures alerts, not just for the new code repo leakage events, and it is not mentioned in the release notes.
Recommendation:
Add a second changelog entry describing the severity fix:
- version: "2.9.0"
changes:
- description: Add support for code repo leakage event in playbook alert data stream.
type: enhancement
link: https://github.com/elastic/integrations/pull/20847
- description: Map the `Moderate` playbook alert priority to `event.severity`.
type: bugfix
link: https://github.com/elastic/integrations/pull/20847🤖 AI-Generated Review | Vera Review Bot - v0.2.6 | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
| changes: | ||
| - description: Add support for code repo leakage event in playbook alert data stream. | ||
| type: enhancement | ||
| link: https://github.com/elastic/integrations/pull/1 |
There was a problem hiding this comment.
Severity: 🟡 Medium confidence: high path: packages/ti_recordedfuture/changelog.yml:6
Changelog links a different PR number
Details
This changelog entry's link: points at pull/1, but it was added in PR #20847. It is likely a leftover template placeholder or a copy from another PR.
Recommendation:
Point each added changelog entry's link at this PR:
link: https://github.com/elastic/integrations/pull/20847🤖 AI-Generated Review | Vera Review Bot - v0.2.6 | 📚 Knowledge base: integration-skills
⚠️ Automated review — verify suggestions before applying.
Review summaryIssues found across the latest commits 1b9d923 — 2 medium, 2 low
🤖 AI-Generated Review | Vera Review Bot - v0.2.6 | 📚 Knowledge base: integration-skills
|
1 participant
Proposed commit message
Checklist
changelog.ymlfile.How to test this PR locally
Related issues