Skip to content

SdSadat/NetSight

Repository files navigation

NetSight

Universal API discovery tool for Chrome DevTools, capture, analyze, and catalog network traffic from any web application.

NetSight sits inside Chrome DevTools and watches every API call your page makes. Define what you're looking for with declarative schemas, and NetSight scores, matches, and exports the endpoints you care about. No vendor lock-in, works on any website.


Features

Live Traffic Capture, Intercept all XHR/Fetch requests in real time. Sort by method, status, URL, duration, or size. Expand any row to inspect request/response bodies and copy them as cURL or Fetch code.

Smart Schema Matching, Define what endpoints you're looking for using declarative JSON schemas. NetSight scores every captured request using a composite engine (URL patterns, HTTP method, request body keys, response structure) and ranks matches by confidence.

Multi-Format Export, Export any discovered endpoint as API Spec JSON, cURL, JavaScript Fetch, TypeScript interfaces, or OpenAPI 3.0.

Configurable Capture Filters, Control exactly what gets captured: domain includes/excludes, path pattern matching, MIME type filtering, and built-in presets for REST, GraphQL, and same-origin traffic.

Auto-Crawl, Spider a website automatically using BFS link extraction. NetSight navigates pages, waits for network idle, captures all API calls, and builds a comprehensive API map without manual browsing.


Installation

git clone https://github.com/SdSadat/NetSight
cd NetSight
npm install
npm run build
  1. Open Chrome and go to chrome://extensions
  2. Enable Developer mode (top-right toggle)
  3. Click Load unpacked and select the project root
  4. Open DevTools on any page (F12), find the NetSight tab
npm run dev    # watch mode, rebuilds on file changes
npm run build  # production build

Usage

1. Capture Traffic

Open the Traffic tab. NetSight records automatically. Browse your target site, every JSON API call appears in the table. Use the search bar and method filters to narrow results. Click any row to expand it and view the full response, or copy it as cURL/Fetch.

2. Define Schemas

Switch to the Schemas tab and paste a DiscoverySchema JSON. Example, find user profile endpoints:

{
  "name": "User Profile",
  "description": "Endpoint returning user profile data",
  "method": "GET",
  "urlHints": {
    "contains": ["/user", "profile"],
    "excludes": ["/logout", "/session"]
  },
  "responseHints": {
    "requiredKeys": ["email", "name"],
    "keyValuePatterns": [
      { "key": "email", "type": "string", "contains": "@" }
    ],
    "minKeys": 3
  },
  "expectedOutput": {
    "responsePath": "$",
    "isArray": false
  }
}

3. View Matches

The Matches tab shows all captured requests scored against your schemas. Each match displays a confidence level (high/medium/low), a score breakdown across five categories, and the full request/response data.

4. Export

Click Export on any match to open the multi-format dialog. Pick a format, copy, and use.

5. Configure Filters

The Settings tab lets you fine-tune capture:

Preset What it captures
All Traffic All JSON API calls
REST APIs Paths containing /api/, /v1/, /v2/, /rest/
GraphQL /graphql requests only
Same Origin Requests to the current domain only

You can also set custom domain/path includes and excludes, adjust the buffer size (50–5000 entries), and define template parameters for dynamic URL values on export.


Schema Reference

interface DiscoverySchema {
  name: string;
  description: string;
  method: "GET" | "POST" | "ANY";

  urlHints: {
    contains?: string[];     // Substrings that MUST appear (AND logic)
    pattern?: string;        // Regex pattern to match
    excludes?: string[];     // Disqualify if URL contains these
  };

  requestBodyHints?: {
    containsKeys?: string[];                       // Dot-notation paths that must exist
    containsValues?: Record<string, unknown>;      // Key-value pairs to match
  };

  responseHints: {
    requiredKeys?: string[];     // Keys that must exist (recursive search, depth 5)
    keyValuePatterns?: Array<{
      key: string;               // Dot-path (e.g. "data.user.email")
      type?: "string" | "number" | "boolean" | "array" | "object";
      contains?: string;         // Substring match for string values
    }>;
    isArrayAt?: string;          // Path where an array is expected ("$" or "results")
    minKeys?: number;            // Minimum keys in response object
  };

  expectedOutput: {
    responsePath: string;        // Path to the data ("$" for root)
    isArray: boolean;
  };
}

Scoring Breakdown

Category Max What it checks
URL 25 Substring matches, regex pattern
Method 10 HTTP method match
Request Body 15 Key presence, value matching
Response Keys 25 Required keys (recursive), array presence, key count
Response Values 25 Path-based type and value checks
Total 100

Confidence: High ≥ 80 · Medium ≥ 50 · Low < 50


Example Schemas

REST, Paginated List

{
  "name": "Paginated Users",
  "description": "List endpoint with pagination metadata",
  "method": "GET",
  "urlHints": { "contains": ["/users"], "excludes": ["/auth"] },
  "responseHints": {
    "requiredKeys": ["total", "page"],
    "isArrayAt": "data",
    "keyValuePatterns": [
      { "key": "data", "type": "array" },
      { "key": "total", "type": "number" }
    ]
  },
  "expectedOutput": { "responsePath": "data", "isArray": true }
}

GraphQL, Named Query

{
  "name": "GraphQL User Query",
  "description": "GraphQL query returning user data",
  "method": "POST",
  "urlHints": { "contains": ["/graphql"] },
  "requestBodyHints": {
    "containsKeys": ["query"],
    "containsValues": { "operationName": "GetUser" }
  },
  "responseHints": {
    "requiredKeys": ["data"],
    "keyValuePatterns": [
      { "key": "data.user.id", "type": "string" }
    ]
  },
  "expectedOutput": { "responsePath": "data.user", "isArray": false }
}

Authentication, Token Response

{
  "name": "Auth Token",
  "description": "Authentication endpoint returning access token",
  "method": "POST",
  "urlHints": { "contains": ["/auth", "token"], "excludes": ["/refresh"] },
  "responseHints": {
    "requiredKeys": ["access_token"],
    "keyValuePatterns": [
      { "key": "access_token", "type": "string" },
      { "key": "token_type", "type": "string", "contains": "bearer" }
    ],
    "minKeys": 2
  },
  "expectedOutput": { "responsePath": "$", "isArray": false }
}

Architecture

Chrome DevTools Page
    │
    ▼
NetworkCapture (chrome.devtools.network.onRequestFinished)
    │  filtered by CaptureConfig (domains, paths, MIME types)
    ▼
CapturedEntry { url, method, body, response, status, size, duration }
    │
    ├──▶ TrafficTable (live sortable/filterable table)
    │
    ▼
MatchEngine (scores entry against all DiscoverySchemas)
    │  composite: URL(25) + Method(10) + Body(15) + Keys(25) + Values(25)
    ▼
MatchResult { score, confidence, entry, schema }
    │
    ▼
Export (ApiSpec | cURL | Fetch | TypeScript | OpenAPI 3.0)

Project Structure

src/
├── capture/
│   ├── harParser.ts         # URL filtering + HAR entry parsing
│   └── networkCapture.ts    # Capture manager singleton
├── crawler/
│   └── autoCrawler.ts       # BFS page spider
├── engine/
│   ├── bodyScorer.ts        # Request body scoring
│   ├── keySearch.ts         # Recursive key finder
│   ├── matcher.ts           # Score orchestration
│   ├── responseScorer.ts    # Response key/value scoring
│   └── urlScorer.ts         # URL pattern scoring
├── panel/
│   ├── App.tsx              # Root with tab layout
│   ├── panel.css            # Dark theme styles
│   ├── store.ts             # React Context state
│   ├── components/
│   │   ├── AutoCrawl.tsx
│   │   ├── ErrorBoundary.tsx
│   │   ├── ExportDialog.tsx
│   │   ├── Icons.tsx        # SVG icons + empty states
│   │   ├── MatchDetail.tsx
│   │   ├── MatchList.tsx
│   │   ├── SchemaInput.tsx
│   │   ├── Settings.tsx
│   │   └── TrafficTable.tsx
│   └── hooks/
│       ├── useCapture.ts
│       ├── useConfig.ts
│       ├── useCrawler.ts
│       └── useMatchEngine.ts
├── types/
│   ├── config.ts            # CaptureConfig + filter presets
│   ├── schema.ts            # Core domain types
│   └── spec.ts              # Export format types
└── utils/
    ├── exportFormats.ts     # cURL, Fetch, TS, OpenAPI generators
    └── jsonPath.ts          # Dot-path accessor

License

MIT

About

Universal API discovery tool for Chrome DevTools, capture, analyze, and catalog network traffic from any web application

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

 
 
 

Contributors

Languages