Securely Triggering a Private Jenkins Using GitHub Webhooks, API Gateway (Public), and Lambda (Private)

In real production setups, one rule is very clear:
Jenkins should never be exposed to the internet.

At the same time, GitHub Webhooks can only hit public endpoints.

So the obvious question becomes:

How do we accept GitHub webhooks from the public internet and still trigger a private Jenkins safely?

This is a problem I recently worked on, and in this post I’ll walk through the exact approach that worked in production, including one important API Gateway setting that breaks the entire flow if you miss it.

The Idea Behind the Architecture

The core idea is simple:

Let API Gateway be the only public-facing component

Keep Lambda and Jenkins private

Validate everything before Jenkins is triggered

Here’s the flow:

GitHub (Public)
   |
   | HTTPS Webhook
   v
API Gateway (Public REST API)
   |
   | Lambda Proxy Integration
   v
AWS Lambda (Private – inside VPC)
   |
   | Internal HTTPS
   v
Jenkins (Private – EC2 / VPC)

This way:

Jenkins never sees internet traffic

Lambda becomes the security gate

Only verified GitHub events trigger builds

What We’re Using

This setup uses:

API Gateway (REST API) as the public webhook endpoint
AWS Lambda inside a VPC for validation and logic
Private Jenkins running inside the same VPC
GitHub HMAC signature verification to ensure payload integrity
Nothing fancy — just the right components in the right places.

Step 1: Set Up Jenkins (Private)

First, Jenkins.
Jenkins should:
Run on EC2 or a VM inside a VPC
Be placed in a private subnet
Have no public IP
Security Group rules should look like this:
Inbound: only allow traffic from Lambda’s security group
Outbound: allow HTTPS if Jenkins needs external access
At this point, Jenkins is completely isolated from the internet, which is exactly what we want.

Step 2: Create the Lambda Function (Private)

Next, create the Lambda function that will sit between GitHub and Jenkins.
Basic configuration:
Runtime: Python 3.10
Timeout: 10–15 seconds
Memory: 128 MB
Then attach Lambda to:
Private subnets
A security group that allows outbound HTTPS
This is important because Lambda needs to reach private Jenkins over the internal network.

Step 3: Create API Gateway (Public Entry Point)

Now we need something public for GitHub to call.

⚠️ Use REST API, not HTTP API.

This matters because REST API gives you full control over request validation and headers.

Steps:

API Gateway → Create API
Choose REST API
Create a resource (for example /)
Add a POST method
This API will be the only thing exposed to the internet.

Step 4: Connect API Gateway to Lambda

In the POST method’s Integration Request:
Integration type: Lambda Function
Select your Lambda
Enable Lambda Proxy Integration
Set timeout to 29 seconds
Deploy the API (for example to stage dev)
Lambda Proxy Integration is important because it forwards:

–

Headers

Body
Raw request payload
without modifying anything.

Step 5: Method Request Configuration (This Is the Gotcha)

This is the step that caused me trouble initially, and it’s where many implementations fail.

Go to:

API Gateway
→ Resources
→ POST
→ Method Request

Set the following:

Authorization: None

Request Validator:
Validate body, query string parameters, and headers

API Key Required: No

Why this matters:

Without this, API Gateway may drop headers

X-Hub-Signature-256 never reaches Lambda

GitHub signature verification fails even if your code is correct

After enabling this:
👉 Redeploy the API

This one setting fixed the entire issue.

Step 6: Configure the GitHub Webhook

In your GitHub repository:

Settings → Webhooks → Add webhook

Use:

Payload URL:

https://api-id.execute-api.region.amazonaws.com/dev/

Content type:

application/json

Secret:

demo-github-secret

Events:

Push events

GitHub will now send:

The raw JSON payload
The X-Hub-Signature-256 header
What the Lambda Actually Does
Lambda acts as a security gate between GitHub and Jenkins.
It does four things:
Reads the raw webhook payload
Verifies the GitHub signature

Checks the branch

Triggers Jenkins privately

Let’s look at the code and logic.

Lambda Code (Production Ready)

import json
import hmac
import hashlib
import base64
import urllib.parse
import requests
from requests.auth import HTTPBasicAuth

GITHUB_SECRET = b"demo-github-secret"

JENKINS_URL = "https://jenkins.example.com"
JENKINS_USER = "demo-user"
JENKINS_TOKEN = "demo-api-token"

JOB_URL = f"{JENKINS_URL}/job/demo-pipeline/buildWithParameters"

This is basic setup:

GitHub secret for verification

Jenkins URL and API token for authentication

Extracting the Raw Payload (Most Important Part)

def extract_payload(event):
    headers = {k.lower(): v for k, v in (event.get("headers") or {}).items()}
    body = event.get("body", "")

Headers are normalized to lowercase so we don’t run into case-sensitivity issues.

if event.get("isBase64Encoded"):
        raw_body = base64.b64decode(body)
    else:
        raw_body = body.encode("utf-8")

GitHub signs the raw bytes, not parsed JSON.
This step preserves the payload exactly as GitHub sent it.

if raw_body.startswith(b"payload="):
        raw_body = urllib.parse.unquote_to_bytes(
            raw_body.decode().replace("payload=", "", 1)
        )

This handles older webhook formats and makes the function more robust.

Verifying the GitHub Signature

def verify_signature(raw_body, headers):
    signature = headers.get("x-hub-signature-256")
    if not signature:
        return False

If the signature header is missing, the request is rejected immediately.

expected = "sha256=" + hmac.new(
        GITHUB_SECRET,
        raw_body,
        hashlib.sha256
    ).hexdigest()

We recreate the signature using the same raw bytes.

 return hmac.compare_digest(signature, expected)

This securely compares both values and prevents timing attacks.

Main Lambda Handler

def lambda_handler(event, context):
    raw_body, headers = extract_payload(event)

Always extract and verify first. Never parse JSON before validation.


    if not verify_signature(raw_body, headers):
        return {"statusCode": 401, "body": "Invalid GitHub signature"}

Invalid requests are blocked here. Jenkins never sees them.

    payload = json.loads(raw_body.decode("utf-8"))

Only now is it safe to parse the JSON.

ref = payload.get("ref", "")
    if not ref.endswith("/main"):
        return {"statusCode": 200, "body": "Not main branch"}

This ensures Jenkins is triggered only for the main branch.

Triggering Jenkins (Privately)

 session = requests.Session()
    session.auth = HTTPBasicAuth(JENKINS_USER, JENKINS_TOKEN)

We authenticate using a Jenkins API token.

crumb = session.get(f"{JENKINS_URL}/crumbIssuer/api/json").json()
    crumb_header = {crumb["crumbRequestField"]: crumb["crumb"]}

This handles Jenkins CSRF protection.

response = session.post(
        JOB_URL,
        headers=crumb_header,
        params={"BRANCH": "main"},
        timeout=10
    )

The pipeline is triggered over the internal network.

 return {
        "statusCode": response.status_code,
        "body": "Jenkins pipeline triggered"
    }

Understanding the Lambda Code

Lambda acts as a security boundary between GitHub and Jenkins.

It performs four tasks:

Extracts the raw webhook payload

Verifies the GitHub signature

Filters events (only main branch)

Triggers private Jenkins

Lambda Code (Production-Ready)

import json
import hmac
import hashlib
import base64
import urllib.parse
import requests
from requests.auth import HTTPBasicAuth

GITHUB_SECRET = b"demo-github-secret"

JENKINS_URL = "https://jenkins.example.com"
JENKINS_USER = "demo-user"
JENKINS_TOKEN = "demo-api-token"

JOB_URL = f"{JENKINS_URL}/job/demo-pipeline/buildWithParameters"


def extract_payload(event):
    headers = {k.lower(): v for k, v in (event.get("headers") or {}).items()}
    body = event.get("body", "")

    if event.get("isBase64Encoded"):
        raw_body = base64.b64decode(body)
    else:
        raw_body = body.encode("utf-8")

    if raw_body.startswith(b"payload="):
        raw_body = urllib.parse.unquote_to_bytes(
            raw_body.decode().replace("payload=", "", 1)
        )

    return raw_body, headers


def verify_signature(raw_body, headers):
    signature = headers.get("x-hub-signature-256")
    if not signature:
        return False

    expected = "sha256=" + hmac.new(
        GITHUB_SECRET,
        raw_body,
        hashlib.sha256
    ).hexdigest()

    return hmac.compare_digest(signature, expected)


def lambda_handler(event, context):
    raw_body, headers = extract_payload(event)

    if not verify_signature(raw_body, headers):
        return {"statusCode": 401, "body": "Invalid GitHub signature"}

    payload = json.loads(raw_body.decode("utf-8"))

    ref = payload.get("ref", "")
    if not ref.endswith("/main"):
        return {"statusCode": 200, "body": "Not main branch"}

    session = requests.Session()
    session.auth = HTTPBasicAuth(JENKINS_USER, JENKINS_TOKEN)

    crumb = session.get(f"{JENKINS_URL}/crumbIssuer/api/json").json()
    crumb_header = {crumb["crumbRequestField"]: crumb["crumb"]}

    response = session.post(
        JOB_URL,
        headers=crumb_header,
        params={"BRANCH": "main"},
        timeout=10
    )

    return {
        "statusCode": response.status_code,
        "body": "Jenkins pipeline triggered"
    }

Final Takeaways

API Gateway is the only public component

Lambda and Jenkins remain private

GitHub signatures must be verified using raw payload bytes

Method Request validation in API Gateway is critical

This pattern works reliably in real production environments

Leave a Reply Cancel reply