Security Review in Practice

1Demonstration Overview

This article puts the entire series into practice. We'll run the complete security review workflow on a real open-source application, showing every step from SAST scanning through AI triage, deep review, and remediation.

The series Introduction cataloged sixteen failure modes that affect agentic workflows—and this demonstration is where you can see the structural defenses against them in action. Verbatim evidence requirements catch phantom grounding before fabricated findings reach the report. The checklist-validator-auditor triple layer catches completeness gaps that a single-pass review would miss. Checkpoint verification between phases interrupts error compounding before a small upstream mistake corrupts everything downstream. Where the taxonomy describes the failure modes in the abstract, this walkthrough shows what they look like—and what catching them looks like—in practice.

The Workflow

The workflow has two parallel branches that merge at findings.md:

Key insight: The checklist is a reusable methodology template. SAST findings don't feed into the checklist—both branches produce findings that merge in findings.md.

Target Application

Tools

2Environment Setup

Install Semgrep

# Install Semgrep
pip install semgrep

# Verify installation
semgrep --version

Actual result: Semgrep 1.149.0 installed successfully.

Clone Target Repository

# Clone the RealWorld Express implementation (original stack fork)
git clone https://github.com/123MwanjeMike/node-express-realworld-example-app.git target-app
cd target-app

# Verify codebase structure
find . -name "*.js" -not -path "./node_modules/*" | xargs wc -l

Actual result: 13 JavaScript files cloned. Structure verified:

• app.js — Main application entry
• config/ — Configuration including JWT secret
• models/ — Mongoose models (User, Article, Comment)
• routes/api/ — Express route handlers (users, articles, profiles, tags)

Verify Subagents

# Subagents should be installed from Part 3
ls ~/.claude/agents/

# Required subagents:
# - sast-triage.md
# - security-scanner.md
# - checklist-validator.md
# - findings-writer.md
# - self-auditor.md
# - fix-implementer.md (from Part 4)
# - fix-verifier.md (from Part 4)

Create Working Directory

# Create directory for review artifacts
mkdir security-review && cd security-review

# Files we'll generate:
# - sast-results.json
# - sast-triage-report.md
# - checklist.md
# - plan.md
# - findings.md
# - remediation-log.md

3Seed Test Vulnerabilities

To ensure a meaningful demonstration with predictable results, we inject specific vulnerabilities into the codebase. This gives us guaranteed findings across different categories: SAST-detectable patterns, AI-only logic flaws, and false positive triggers.

Category 1: SAST-Detectable (Pattern-Based)

These vulnerabilities follow patterns that Semgrep rules can match:

SEED-01: NoSQL Injection

File: routes/api/users.js

// SEED-01: NoSQL injection via $where operator
// Add this route after the existing user routes

router.get('/search', auth.optional, function(req, res, next) {
  var query = req.query.q;
  // VULNERABLE: User input directly in $where clause
  User.find({ $where: "this.username.includes('" + query + "')" })
    .then(function(users) {
      return res.json({ users: users.map(u => u.toProfileJSONFor()) });
    }).catch(next);
});

SEED-02: Hardcoded Secret

File: config/index.js

// SEED-02: Hardcoded JWT secret (modify existing secret line)
// Change from: secret: process.env.SECRET || 'secret'
// Change to:
module.exports = {
  secret: process.env.SECRET || 'super_secret_jwt_key_12345',
  // ... rest of config
};

SEED-03: Dangerous eval()

File: routes/api/articles.js

// SEED-03: eval() with user input
// Add this route for "advanced filtering"

router.get('/filter', auth.optional, function(req, res, next) {
  var filterExpr = req.query.expr;
  // VULNERABLE: eval with user-controlled input
  var filterFn = eval('(function(article) { return ' + filterExpr + '; })');
  
  Article.find({}).then(function(articles) {
    var filtered = articles.filter(filterFn);
    return res.json({ articles: filtered });
  }).catch(next);
});

SEED-04: Regex DoS

File: models/User.js

// SEED-04: Catastrophic backtracking regex
// Add this method to UserSchema

UserSchema.methods.validateBio = function() {
  // VULNERABLE: Regex with catastrophic backtracking
  var bioPattern = /^([a-zA-Z]+)*$/;
  return bioPattern.test(this.bio);
};

Category 2: AI-Only (Requires Understanding Intent)

These vulnerabilities require understanding what the code should do, not just pattern matching:

SEED-05: IDOR on Article Update

File: routes/api/articles.js

// SEED-05: Missing ownership check on article update
// Replace the existing PUT /articles/:slug handler with:

router.put('/:article', auth.required, function(req, res, next) {
  // VULNERABLE: No check that req.auth.id === article.author
  // Any authenticated user can update any article
  
  if(typeof req.body.article.title !== 'undefined'){
    req.article.title = req.body.article.title;
  }
  if(typeof req.body.article.description !== 'undefined'){
    req.article.description = req.body.article.description;
  }
  if(typeof req.body.article.body !== 'undefined'){
    req.article.body = req.body.article.body;
  }

  req.article.save().then(function(article){
    return res.json({article: article.toJSONFor(req.auth)});
  }).catch(next);
});

SEED-06: IDOR on Comment Deletion

File: routes/api/articles.js

// SEED-06: Missing ownership check on comment deletion
// Replace the existing DELETE /articles/:slug/comments/:comment handler:

router.delete('/:article/comments/:comment', auth.required, function(req, res, next) {
  // VULNERABLE: No check that req.auth.id === comment.author
  // Any authenticated user can delete any comment
  
  req.article.comments.remove(req.comment._id);
  req.article.save()
    .then(function() {
      Comment.find({_id: req.comment._id}).remove().exec();
      res.sendStatus(204);
    }).catch(next);
});

SEED-07: Mass Assignment on User Profile

File: routes/api/users.js

// SEED-07: Mass assignment vulnerability
// Replace the existing PUT /user handler with:

router.put('/user', auth.required, function(req, res, next) {
  User.findById(req.auth.id).then(function(user) {
    if(!user){ return res.sendStatus(401); }

    // VULNERABLE: Spreads all user-provided fields without filtering
    // Attacker can set admin: true, role: 'admin', etc.
    Object.assign(user, req.body.user);

    return user.save().then(function() {
      return res.json({user: user.toAuthJSON()});
    });
  }).catch(next);
});

SEED-08: Favorite Count Manipulation

File: routes/api/articles.js

// SEED-08: Direct manipulation of computed field
// Add this route:

router.put('/:article/favoritesCount', auth.required, function(req, res, next) {
  // VULNERABLE: Allows direct setting of favoritesCount
  // This should be a computed value, not user-settable
  
  req.article.favoritesCount = req.body.count;
  req.article.save().then(function(article) {
    return res.json({article: article.toJSONFor(req.auth)});
  }).catch(next);
});

Category 3: False Positive Triggers

These patterns look dangerous to SAST but are safe in context—testing AI triage accuracy:

SEED-09: SQL-like Syntax in Logging (Safe)

File: routes/api/articles.js

// SEED-09: SQL-like syntax that's actually just a log message
// Add to the GET /articles handler:

router.get('/', auth.optional, function(req, res, next) {
  // This is just a log message, not executed as SQL
  console.log("Query: SELECT * FROM articles WHERE author = '" + 
              (req.query.author || 'anonymous') + "' -- for debugging");
  
  // ... rest of existing handler
});

SEED-10: Validated Input Used in Dangerous Pattern (Safe)

File: routes/api/tags.js

// SEED-10: Looks dangerous but input is validated
// Add this route:

router.get('/search', function(req, res, next) {
  var tag = req.query.tag;
  
  // Input is strictly validated - alphanumeric only, max 20 chars
  if (!/^[a-zA-Z0-9]{1,20}$/.test(tag)) {
    return res.status(400).json({ error: 'Invalid tag format' });
  }
  
  // SAST may flag this, but it's safe due to validation above
  var searchPattern = new RegExp(tag, 'i');
  
  Article.find({ tagList: searchPattern }).then(function(articles) {
    return res.json({ articles: articles });
  }).catch(next);
});

Apply the Seeds

# Create a patch file with all seed vulnerabilities
cat > seed-vulnerabilities.patch << 'EOF'
[PLACEHOLDER: Generated patch file from the above changes]
EOF

# Apply the patch
cd node-express-realworld-example-app
git apply ../seed-vulnerabilities.patch

# Commit the seeded version for reproducibility
git add -A
git commit -m "DEMO: Seed vulnerabilities for security review demonstration"

4SAST Baseline Scan

With vulnerabilities seeded, we establish a baseline of pattern-detected issues using Semgrep.

Running the Scan

# Run Semgrep with security-focused rulesets
# Note: p/owasp ruleset was unavailable at time of testing
semgrep scan \
  --config p/security-audit \
  --config p/nodejs \
  --json \
  --output security-review/sast-results.json \
  target-app

# Human-readable summary
semgrep scan \
  --config p/security-audit \
  --config p/nodejs \
  target-app

SAST Results

SAST Findings Detail

Analysis: Seeded vs Detected

5AI Triage of SAST Findings

Now we use the sast-triage subagent to classify each finding by analyzing context that Semgrep couldn't see.

Triage Prompt

Use the sast-triage subagent to analyze the Semgrep findings in 
sast-results.json.

For each finding:
1. Locate the flagged code and read 30+ lines of context
2. Trace data flow backward to the source
3. Trace data flow forward to the sink
4. Evaluate mitigations (framework protections, validation, architecture)
5. Classify as:
   - TRUE_POSITIVE: Exploitable vulnerability
   - FALSE_POSITIVE: Safe in context (document why)
   - NEEDS_INVESTIGATION: Cannot determine statically

Write detailed triage results to sast-triage-report.md

Triage Results

Triage Summary by Priority

Example: True Positive (Critical)

// routes/api/articles.js:35
var filterExpr = req.query.expr;  // User-controlled input
var filterFn = eval('(function(article) { return ' + filterExpr + '; })');

Source Analysis: User input from req.query.expr flows directly into the code.

Sink Analysis: eval() executes arbitrary JavaScript on the server.

Attack Scenario:

GET /api/articles/filter?expr=process.exit(1))%3B//

This crashes the server. More dangerous payloads could execute system commands, read files, or establish reverse shells.

Mitigations Found: NONE. Code explicitly comments "VULNERABLE".

Conclusion: Critical RCE vulnerability allowing complete server compromise.

Example: Needs Investigation (Contextual)

Source Analysis: Session config doesn't set explicit domain property.

Sink Analysis: Without explicit domain, browser defaults to host-only (more restrictive).

Context Required: For single-domain apps, the default is actually MORE secure. For multi-subdomain apps sharing sessions, explicit configuration is needed.

Conclusion: Classification depends on deployment architecture—not a universal vulnerability.

Additional Findings Noted During Triage

The AI triage also identified vulnerabilities NOT in the SAST report:

• IDOR on article update (missing ownership check)
• IDOR on comment delete (missing ownership check)
• Business logic flaw (direct favoritesCount manipulation)
• Hardcoded JWT secret fallback in config/index.js

These will be formally documented during AI Deep Review (Phase 6).

Document TRUE_POSITIVEs

8 TRUE_POSITIVEs confirmed from SAST triage. These will be merged with AI-discovered findings in findings.md after the deep review phase.

6Generate and Validate Checklist

Now we generate a security checklist—a reusable methodology template that doesn't depend on SAST output. The checklist defines what to check; findings (from both SAST triage and AI review) document what we found.

Generate Checklist

Read templates/CHECKLIST_TEMPLATE.md and generate a security review 
checklist for this Node.js/Express/MongoDB application.

Stack details:
- Express REST API with JWT authentication
- MongoDB with Mongoose ODM
- Features: users, articles, comments, favorites, follows

Verbosity: STANDARD

Write to checklist.md

Note: The checklist is a reusable methodology template. It doesn't include SAST findings—those go directly to findings.md via the parallel SAST branch.

Validate Checklist

Use the checklist-validator subagent to review checklist.md.

Find gaps by checking against:
- OWASP Top 10
- CWE Top 25
- Node.js/Express-specific security concerns
- MongoDB injection patterns

Merge valid additions with [VALIDATION] tags.

Checklist Summary

Key Validation Gaps Identified

7AI Deep Review

Execute the security-scanner subagent for each checklist category, focusing on issues that pattern matching cannot detect.

Execution Prompt

Execute the security review using the subagent workflow:

1. Create plan.md from checklist.md, ordering by severity
2. For each checklist category:
   a. Use security-scanner subagent to examine the codebase
   b. Focus on authorization checks, business logic, data flow
   c. Use findings-writer subagent to document issues
   d. Update plan.md with completion status

Focus particularly on:
- IDOR potential in article/comment endpoints
- Authorization checks (or lack thereof)
- Business logic in favorites/follows

3. Use self-auditor subagent to verify completeness

AI-Discovered Vulnerabilities

Complete Findings Table

Seeded Vulnerabilities - Detection Results

*SEED-04 (ReDoS) requires specialized regex analysis; not in high-priority review categories.

Sample AI-Only Finding: IDOR on Article Update

// VULNERABLE: No check that req.payload.id === article.author
router.put('/:article', auth.required, function(req, res, next) {
  // Any authenticated user can update ANY article!
  if(typeof req.body.article.title !== 'undefined'){
    req.article.title = req.body.article.title;
  }
  // ... updates proceed without ownership verification
});

Attack Scenario: User A creates article. User B (any authenticated user) sends PUT request to modify User A's article. No 403 is returned—the article is updated.

Impact: Complete loss of data integrity. Any user can modify any article's title, description, and body.

Why SAST Missed It: There's no "dangerous pattern" here—the code is syntactically valid. Detecting this requires understanding that updates to owned resources SHOULD verify ownership.

Recommended Fix:

if (req.article.author._id.toString() !== req.payload.id.toString()) {
  return res.sendStatus(403);
}
// Then proceed with updates

Self-Audit Results

The self-auditor verified completeness against the 36-item checklist. All high-priority categories were reviewed. SEED-04 (ReDoS) remained undetected as it requires specialized regex analysis not covered by standard security scanner patterns.

8SAST vs AI: Side-by-Side Comparison

With both analyses complete, we can compare what each approach found.

Detection Rate Comparison

What Each Approach Found

Summary Statistics

9Remediation Demonstration

We demonstrate the remediation workflow on one finding from each detection category: SAST-detected and AI-detected.

SAST-Detected Fix: eval() RCE (F-001 / SEED-03)

Before (Vulnerable)

router.get('/filter', auth.optional, function(req, res, next) {
  var filterExpr = req.query.expr;
  // VULNERABLE: eval with user-controlled input
  var filterFn = eval('(function(article) { return ' + filterExpr + '; })');
  // ...
});

After (Fixed)

router.get('/filter', auth.optional, function(req, res, next) {
  var filterType = req.query.type;
  var filterValue = req.query.value || '';

  // Predefined safe filters - no user code execution
  var safeFilters = {
    'hasTag': function(article) { /* safe implementation */ },
    'minFavorites': function(article) { /* safe implementation */ },
    'recent': function(article) { /* safe implementation */ }
  };

  var filterFn = safeFilters[filterType];
  if (!filterFn) {
    return res.status(400).json({ error: 'Invalid filter type' });
  }
  // ...
});

Fix Reasoning: Removed eval() entirely. Replaced with predefined safe filter functions. User input is treated as data, not code.

AI-Detected Fix: NoSQL Injection (F-002 / SEED-01)

Before (Vulnerable)

router.get('/search', auth.optional, function(req, res, next) {
  var query = req.query.q;
  // VULNERABLE: User input directly in $where clause
  User.find({ $where: "this.username.includes('" + query + "')" })
    .then(function(users) {
      return res.json({ users: users.map(u => u.toProfileJSONFor()) });
    }).catch(next);
});

After (Fixed)

router.get('/search', auth.optional, function(req, res, next) {
  var query = req.query.q || '';
  // SECURITY FIX: Replace $where with safe $regex operator
  var sanitizedQuery = query.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
  User.find({ username: { $regex: sanitizedQuery, $options: 'i' } })
    .then(function(users) {
      return res.json({ users: users.map(u => u.toProfileJSONFor()) });
    }).catch(next);
});

Fix Reasoning: Replaced dangerous $where (executes JavaScript) with safe $regex (pattern matching only). Added regex character escaping to prevent regex injection.

AI-Detected Fix: IDOR (F-005 / SEED-05)

Before (Vulnerable)

router.put('/:article', auth.required, function(req, res, next) {
  // VULNERABLE: No ownership check
  if(typeof req.body.article.title !== 'undefined'){
    req.article.title = req.body.article.title;
  }
  // ...
});

After (Fixed)

router.put('/:article', auth.required, function(req, res, next) {
  // SECURITY FIX: Added ownership verification
  if (req.article.author._id.toString() !== req.payload.id.toString()) {
    return res.sendStatus(403);
  }

  if(typeof req.body.article.title !== 'undefined'){
    req.article.title = req.body.article.title;
  }
  // ...
});

Fix Reasoning: Added ownership verification before allowing updates. Non-owners receive 403 Forbidden.

Verification Results

Git branch: remediated (based on seeded-vulnerabilities)
Full log: security-review/remediation-log.md

10Lessons Learned

What Worked Well

1. Parallel workflow — SAST and AI review ran independently, findings merged cleanly in findings.md
2. Seeded vulnerabilities — Predictable results enabled clear comparison; 7 of 8 detected by combined workflow
3. Triage value — 20% of findings required contextual analysis (NEEDS_INVESTIGATION); zero false positives confirmed
4. Subagent separation — Each agent had clear responsibility: triage classifies, scanner finds, writer documents
5. Checklist validation — Validator caught 10 gaps including an entire OWASP Top 10 category (A09: Logging)

What Could Improve

1. SAST ruleset coverage — Semgrep's default rulesets missed NoSQL $where injection and ReDoS despite being "pattern-based" issues. Custom rules or additional rulesets (MongoDB-specific, regex security) would improve detection.
2. Specialized analysis phases — ReDoS was the only seeded vulnerability neither SAST nor AI detected. Regex security requires specialized algorithmic analysis (NFA complexity) that pattern matching and general AI review cannot provide.
3. Repository drift — The original documented repository had migrated to TypeScript/Prisma. Using forks requires verification that they match documented stack assumptions.
4. Validation item prioritization — Items added by checklist-validator (like ReDoS) received less execution priority than original items.
5. Unused code analysis — SEED-04's validateBio() was defined but never called. A "dead code with security implications" check could flag latent vulnerabilities.

Unexpected Findings

1. SAST detected only 12.5% of seeded vulnerabilities — Expected ~50% based on "pattern-based" classification. The || fallback pattern for hardcoded secrets is a significant SAST blind spot.
2. Pre-existing vulnerabilities outnumbered seeds — The base application had 6 genuine security issues (session configuration, JWT revocation) that SAST correctly identified.
3. Zero false positives — Both false positive triggers (SEED-09, SEED-10) were correctly NOT flagged by SAST, contrary to expectation that they would trigger and require triage dismissal.

Key Takeaways

11Frequently Asked Questions