← Back to Home

Threat Intelligence Platform (TIP)

IOC Enrichment, Blocking & Release

A production module inside a Security Command Center (SCC) platform that aggregates indicators of compromise from many intelligence sources, deduplicates and enriches them into a single scored feed, manages their block / release lifecycle, and keeps the enterprise SIEM and firewalls automatically in sync — exposing the result as standards-based threat feeds.

#Threat Intelligence#IOC Management#Threat Feed#SIEM Integration#STIX / TAXII#Data Engineering#ELT / OLAP#Security Automation (SOC/SOAR)#Lifecycle Management

// Overview

What it does. A SOC consumes threat intelligence from many providers, but raw feeds are duplicated, inconsistently scored, and not directly actionable. This module turns that firehose into one clean, deduplicated, risk-scored IOC feed (IPs, domains, URLs, hashes) and manages what happens next: which indicators get blocked at the SIEM/firewall, and which are later released (unblocked) when they're no longer a threat or were false positives. High-confidence indicators are pushed to the enterprise SIEM automatically every few minutes, and the curated feed is published in standard formats (STIX/TAXII, CSV, JSON, plaintext) so any downstream security tool can consume it. Analysts work it all from a portal: enrichment dashboards and IOC search, a blocking workflow (including bulk upload), and a release workflow — each with its own stats.

// Background

The challenge

Overlapping provider feeds, a hand-managed SIEM

blocklist, and no lifecycle — coverage always unknown.

01

Duplicated, inconsistent scoring

Same indicator from multiple providers with different

or missing scores; no unified, comparable view.

No single deduplicated source of truth

02

Manual SIEM updates won't scale

Hand-managing blocklists slow, error-prone — stale

orphans lingered after indicators were retired.

Stale entries outlived the threats they blocked

03

No managed lifecycle

Block/unblock decisions lived in side-channels; no

record of what was blocked, why, or when released.

Governance was an afterthought, not a workflow

04

Hard to integrate downstream

Other tools needed STIX/CSV/JSON; bespoke DB

not directly consumable without custom glue.

Every consumer needed a custom integration

Net effect

Analysts spent time managing lists instead of

investigating threats — with no audit trail.

// Solution

IOC pipeline — enrich, govern & sync

Data sources

Commercial threat intel

IOC: IPs · domains · hashes

daily parallel ingest

Open source intelligence

community feeds + bulk uploads

Pipeline

Ingest

Airflow

raw landing

Ingest

Enrich

dedup · score

last-seen

Enrich

Materialize

atomic swap

OLAP + archive

Fact

Lifecycle

block · release

Vue 3 · bulk

Govern

Distribute

SIEM · 15 min

STIX/TAXII · CSV · JSON

Distribute

Airflow · parallel multi-source ingest · dedup + risk scoring · atomic swap · block/release · 15-min SIEM sync

// Impact

SIEM: manual lists → auto sync

Before · by hand

slow · error-prone · stale orphans

After · auto sync

adds new · prunes stale · 15 min

15 min

SIEM sync cycle — always current

Orphaned blocklist entriesauto-pruned
Manual list-wranglingeliminated

Operational impact

Deduplicated, risk-scored feed

noisy providers → one source of truth

Always-current SIEM defenses

15-min delta adds new, prunes stale

Governed block & release lifecycle

auditable workflows + bulk upload

Drop-in interoperability

STIX/TAXII · CSV · JSON feeds

Reliable, consistent dashboards

atomic swap: no partial reads

History stays lean

archive keeps recent N per indicator

Results

IOC types

4

SIEM sync cycle

~15 min

auditable

100%

Export formats

3+

Daily enrichment pipelineautomated
Stale SIEM orphansauto-pruned

Block/release tracked per indicator

keeps history lean over time.

// Tech Stack

LayerTechnologies
OrchestrationApache Airflow — daily TIP DAG (ingest → enrich → fact → archive) + 15-min SIEM-sync DAG, Papermill
Ingestion / enrichmentMultiple commercial & open threat-intel sources, Python requests, pandas, parallel tasks
Data storePostgreSQL — raw landing, deduplicated summary, atomically-swapped OLAP fact table, archive tables, IOC dimension
DistributionSTIX/TAXII 2.1, CSV, JSON, plaintext feed endpoints; enterprise SIEM admin API (bearer auth, batched add/delete)
Backend APIFastAPI, in-memory CacheStore, psycopg / SQLAlchemy
FrontendVue 3, Quasar, Vite, TypeScript, Pinia, TanStack Vue Query, ECharts

// Architecture & Diagrams

1 / 4System Architecture — Simple (high level)
rendering…

Disclaimer: sample visuals may contain anonymized, simulated, or non-production values for presentation purposes.

// Demo

No demo configured yet.