[REQUEST] Ensure external_references' urls are RFC3986 Compliant

[jondricek]

Is your feature request related to a problem? Please describe.

STIX 2.1 requires that the Common Data Type of External Reference (external_references) have their url field be RFC3986 compliant (link to STIX 2.1 reference). Ensuring this in the ATT&CK Data Model would help ensure that URLs stay in compliance with STIX 2.1 requirements.

Describe the solution you'd like

This could be added to the zod schema here: https://github.com/mitre-attack/attack-data-model/blob/main/src/schemas/common/property-schemas/stix-external-references.ts

Describe alternatives you've considered

Alternatively we could not bother with this enhancement since it is STIX 2.1 specific, but if not here, then where!?

Additional context

This has bitten the ATT&CK dataset in the past, e.g. here: mitre-attack/attack-stix-data#16

The referenced issue has some good examples of what was historically included in the STIX bundles before they were manually corrected

[seansica]

The url is already being parsed by Zod's built-in .url() validator — here's how we use it:

export const externalReferenceSchema = z.object({
  source_name: nonEmptyRequiredString,
  description: nonEmptyRequiredString.optional(),
  url: z
    .url({
      error: (issue) =>
        issue.input === null
          ? 'URL cannot be null'
          : 'Invalid URL format. Please provide a valid URL',
    })
    .optional(),

  external_id: nonEmptyRequiredString.optional(),
});

Zod documents how their URL parser works here: https://zod.dev/api?id=urls#urls

To validate any WHATWG-compatible URL:

const schema = z.url();

schema.parse("https://example.com"); // ✅
schema.parse("http://localhost"); // ✅
schema.parse("mailto:noreply@zod.dev"); // ✅
As you can see this is quite permissive. Internally this uses the new URL() constructor to validate inputs; this behavior may differ across platforms and runtimes but it's the mostly rigorous way to validate URIs/URLs on any given JS runtime/engine.

To validate the hostname against a specific regex:

const schema = z.url({ hostname: /^example\.com$/ });
 
schema.parse("https://example.com"); // ✅
schema.parse("https://zombo.com"); // ❌

To validate the protocol against a specific regex, use the protocol param.

const schema = z.url({ protocol: /^https$/ });
 
schema.parse("https://example.com"); // ✅
schema.parse("http://example.com"); // ❌

Web URLs — In many cases, you'll want to validate Web URLs specifically. Here's the recommended schema for doing so:

const httpUrl = z.url({
  protocol: /^https?$/,
  hostname: z.regexes.domain
});

This restricts the protocol to http/https and ensures the hostname is a valid domain name with the z.regexes.domain regular expression:

/^([a-zA-Z0-9](?:[a-zA-Z0-9-]{0,61}[a-zA-Z0-9])?\.)+[a-zA-Z]{2,}$/

To normalize URLs, use the normalize flag. This will overwrite the input value with the normalized URL returned by new URL().

new URL("HTTP://ExAmPle.com:80/./a/../b?X=1#f oo").href
// => "http://example.com/b?X=1#f%20oo"

So the question is, how permissible is RFC 3986 as it relates to the url key in STIX? Is the broader z.url() parser sufficient, or do we need to constrain it to Web URLs only?

STIX 2.1 mandates RFC 3986, which is the generic URI syntax (covers http, https, mailto, urn, tel, ftp, file, etc.), not just Web URLs. Constraining to protocol: /^https?$/ would be stricter than STIX requires and would reject technically-valid external references like mailto: or urn: schemes.

And if we're going to be sticklers, technically WHATWG ≠ RFC 3986. The two specs overlap heavily but diverge at edges (WHATWG normalizes/percent-encodes more aggressively, accepts some inputs RFC 3986 rejects, and vice versa). Practically speaking however, I think z.url() is fine and not worth implementing something custom to achieve perfect compliance.

I think the more important consideration is the fact that, in practice, every external_references[].url in published ATT&CK STIX bundles are http:// or https:// So, if we intend to keep things this way, we might want to scope down our url parser to better suit ATT&CK, though this would technically be a deviation from the STIX spec.