Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
c925214
Track fixer
Zorlin Aug 2, 2025
2ec73ae
New artist routes, fix some serialization issues
Zorlin Aug 3, 2025
86e86cb
Add migrations for 0.33
Zorlin Aug 3, 2025
7835af3
Check fixes
Zorlin Aug 3, 2025
9e73638
Cleanup, attempt categories stuff
Zorlin Aug 4, 2025
9bdc010
Publish dev version
Zorlin Aug 4, 2025
e25482c
Publish 0.1.48-a2
Zorlin Aug 4, 2025
fa41547
Publish 0.1.48-a3
Zorlin Aug 5, 2025
3833905
Publish 0.1.48-a4
Zorlin Aug 5, 2025
a9c7fa5
Publish 0.1.48-a5
Zorlin Aug 5, 2025
631d463
Prevent sync status from hanging the node - release a6
Zorlin Aug 5, 2025
bc50b84
Publish 0.1.48-a7
Zorlin Aug 6, 2025
382ffbb
Fixes for node crash
Zorlin Aug 6, 2025
f2d18c7
Publish 0.1.48-a9 - increase max event listeners for node
Zorlin Aug 6, 2025
ccaabf1
publish a10
Zorlin Aug 6, 2025
9e2c93d
Publish a13
Zorlin Aug 6, 2025
c05742c
Fix directory already exists bug, Docker stuff, add --apiPort flag
Zorlin Aug 8, 2025
4c483a0
Release version 0.1.48-14
Zorlin Sep 30, 2025
6eba484
version
Zorlin Sep 30, 2025
9e70392
version
Zorlin Sep 30, 2025
8b09965
version
Zorlin Sep 30, 2025
555861b
version
Zorlin Sep 30, 2025
9198492
version
Zorlin Sep 30, 2025
e3bc54d
version
Zorlin Sep 30, 2025
4594a28
Fix: Use --useRelays flag for circuit relay client, not --relay
Zorlin Sep 30, 2025
3fa89f7
Add CLAUDE.md documenting ready endpoint requirements and bump versio…
Zorlin Sep 30, 2025
b753cd3
Re-enable sync checking in /ready endpoint
Zorlin Sep 30, 2025
3cbc646
Update pnpm lockfile
Zorlin Sep 30, 2025
500f383
Switch to Debian Trixie base image for Docker
Zorlin Sep 30, 2025
40b4bf0
Fix native module builds in Docker (RIP Nigel)
Zorlin Sep 30, 2025
3942555
Add CLAUDE.md documentation about local vs Docker and force rebuild n…
Zorlin Sep 30, 2025
f4dd7d0
chore: bump lens-sdk
Zorlin Sep 30, 2025
52162d3
Fix Docker build to use local install instead of global
Zorlin Sep 30, 2025
063774f
Update lens-sdk to 0.1.40 with ready check fix
Zorlin Sep 30, 2025
703ab14
0.1.54
Zorlin Sep 30, 2025
c7b9069
Update to lens-sdk 0.1.41 with relay reconnection fixes
Zorlin Sep 30, 2025
cefa667
0.1.55
Zorlin Sep 30, 2025
096bd08
Fix domain announcement to use wss://domain:443 for browser connectivity
Zorlin Sep 30, 2025
63d5a10
0.1.56
Zorlin Sep 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added .docker-entrypoint.sh.swp
Binary file not shown.
16 changes: 16 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
node_modules/
dist/
.git/
.gitignore
*.log
*.md
Dockerfile*
.dockerignore
coverage/
.env
.env.*
*.test.ts
*.spec.ts
__tests__/
.vscode/
.idea/
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,6 @@ yarn-error.log*
coverage/
.env
.env.*
!/.env.example
!/.env.example
debug-run*
Dockerfile.test
60 changes: 60 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
# CLAUDE.md

## CRITICAL: Ready Endpoint Requirements

**THE `/ready` ENDPOINT MUST VERIFY THAT THE NODE HAS SYNCED CONTENT.**

A "ready" check is NOT just "is the service alive". It means:
- The node has connected to peers
- Content replication has occurred
- Data stores contain replicated content
- The node is actually serving the content it should be serving

We care deeply if content is synced. That's the entire point of the ready check.

The challenge is that checking sync status triggers expensive distributed index queries that cause DoS. The solution is to:
1. Cache sync status and only refresh it periodically (not on every `/ready` call)
2. Use the most efficient Peerbit APIs to check replication status
3. Ensure the check completes quickly without exhausting system resources

But the check MUST verify actual content replication, not just peer connectivity.

## CRITICAL: Local Build vs Docker - It Works Locally

**THE LOCAL BUILD WORKS RELIABLY.** When debugging Docker issues:

The site `zb2rhcRzjgGXUxi8PypUJGvk42HXaXQS3VBZ2aYvgAUrDgkZ1` EXISTS and the local build connects, replicates, and runs successfully with these exact parameters:

```bash
rm -r ~/.lens-node/
SITE_ADDRESS=zb2rhcRzjgGXUxi8PypUJGvk42HXaXQS3VBZ2aYvgAUrDgkZ1 \
BOOTSTRAPPERS=/dns4/relay01.eu.riff.cc/tcp/443/wss/p2p/12D3KooWRcsxc5FBG4QU6MoeGmuXi5KbQyYKwZPnkemKV2GZcRvM,/dns4/relay02.us.riff.cc/tcp/443/wss/p2p/12D3KooWGie5X52rmrZ6iDgqrWv8Ecnc3YfUwsopCT6isbVWZHJR,/dns4/relay03.sg.riff.cc/tcp/443/wss/p2p/12D3KooWDDwXRyvibdj1quDsmvmEKPGH2yK2mMEGXonBKc5GPdDh \
./dist/cli/bin.js run -d ~/.lens-node --onlyReplicate --light --useRelays --apiPort 10002 --listenPort 9001
```

Output: `LensService configured.` and the node runs successfully.

If Docker fails with "Failed to load store" but local works, the issue is NOT:
- The site doesn't exist
- The parameters are wrong
- Native modules (if local and Docker both use the same published package)

Look for environment differences: permissions, file paths, network configuration, or Docker-specific issues.

## CRITICAL: Using DeepWiki for Peerbit Research

**DeepWiki is NOT aware of Lens Node specific terminology or implementation details.**

When querying the dao-xyz/peerbit repository via DeepWiki:
- ❌ DO NOT use Lens-specific terms like "Lens node", "Site", "ready check", "featured releases"
- ✅ DO use Peerbit-specific terms like "peer", "SharedLog", "replication", "relay", "DirectStream"
- ❌ DO NOT assume DeepWiki knows about our lens-sdk wrapper or LensService
- ✅ DO ask about core Peerbit concepts like "connectionManager", "DialerOptions", "SeekDelivery"

**Example Queries:**
- ✅ "How does Peerbit handle relay reconnection when relay connections are lost?"
- ❌ "How does Lens handle relay reconnection?"
- ✅ "How does SeekDelivery calculate quorum in Peerbit?"
- ❌ "How does the ready check work with SeekDelivery?"

DeepWiki provides documentation for the **upstream Peerbit library**, not our implementation that wraps it.
25 changes: 18 additions & 7 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,16 +1,27 @@
FROM node:22-alpine
FROM node:22-trixie-slim

WORKDIR /app

ARG TAG
ENV TAG=${TAG:-latest}

# Enable corepack, set PNPM_HOME and ensure pnpm is available
ENV PNPM_HOME=/root/.local/share/pnpm
ENV PATH=$PNPM_HOME:$PATH
RUN corepack enable && corepack prepare pnpm@latest --activate

# Install the CLI
RUN pnpm install -g --dangerously-allow-all-builds @riffcc/lens-node@${TAG}
# Install build dependencies for native modules
RUN apt-get update && apt-get install -y --no-install-recommends \
python3 \
make \
g++ \
&& rm -rf /var/lib/apt/lists/*

# Copy package files and lockfile
COPY package.json pnpm-lock.yaml ./

# Install dependencies locally
RUN pnpm install --frozen-lockfile

# Copy source and build
COPY . .
RUN pnpm build

ENTRYPOINT ["lens-node"]
ENTRYPOINT ["node", "dist/cli/bin.js"]
55 changes: 55 additions & 0 deletions Dockerfile.dev
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Development Dockerfile for building from source
FROM node:18-alpine as builder

WORKDIR /app

# Install pnpm
RUN npm install -g pnpm

# First, build lens-sdk locally
COPY ../lens-sdk /lens-sdk
WORKDIR /lens-sdk
RUN pnpm install --frozen-lockfile && pnpm build

# Now build lens-node
WORKDIR /app

# Copy package files
COPY package.json pnpm-lock.yaml ./

# Link local lens-sdk
RUN pnpm link /lens-sdk

# Install dependencies
RUN pnpm install --frozen-lockfile

# Copy source code
COPY . .

# Build the project
RUN pnpm build

# Runtime stage
FROM node:18-alpine

WORKDIR /app

# Install pnpm in runtime
RUN npm install -g pnpm

# Copy built files and dependencies
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
COPY --from=builder /app/package.json ./package.json

# Create directory for lens-node data
RUN mkdir -p /root/.lens-node

# Make binary executable
RUN chmod +x dist/cli/bin.js

# Expose default ports
EXPOSE 9501 3000

# Default command
CMD ["node", "dist/cli/bin.js", "run"]
74 changes: 74 additions & 0 deletions Dockerfile.full
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Full build Dockerfile that includes lens-sdk from source
# Use the same base image throughout to ensure binary compatibility
FROM node:22-slim as base

# Install pnpm and build tools once in base
RUN npm install -g pnpm && \
apt-get update && \
apt-get install -y python3 make g++ && \
rm -rf /var/lib/apt/lists/*

# Build lens-sdk
FROM base as sdk-builder
WORKDIR /lens-sdk

# Copy lens-sdk files
COPY ./lens-sdk/package.json ./lens-sdk/pnpm-lock.yaml ./
RUN pnpm install --frozen-lockfile

COPY ./lens-sdk/ ./
RUN pnpm build

# Build lens-node
FROM base as node-builder
WORKDIR /app

# Copy lens-node package files
COPY ./lens-node/package.json ./lens-node/pnpm-lock.yaml ./

# Copy built lens-sdk from previous stage
COPY --from=sdk-builder /lens-sdk /lens-sdk

# Install dependencies with local lens-sdk using file: protocol
RUN sed -i 's|"@riffcc/lens-sdk": ".*"|"@riffcc/lens-sdk": "file:/lens-sdk"|' package.json && \
pnpm install --no-frozen-lockfile && \
pnpm rebuild

# Copy lens-node source
COPY ./lens-node/ ./

# Build lens-node
RUN pnpm build

# Runtime stage - same base image!
FROM base

WORKDIR /app

# Copy everything from the build stage
COPY --from=node-builder /app/dist ./dist
COPY --from=node-builder /app/node_modules ./node_modules
COPY --from=node-builder /app/package.json ./package.json
COPY --from=sdk-builder /lens-sdk /lens-sdk

# Rebuild native modules in the runtime container
RUN cd /app && pnpm rebuild

# Create directory for lens-node data
RUN mkdir -p /root/.lens-node

# Make binary executable
RUN chmod +x dist/cli/bin.js

# Copy entrypoint script
COPY ./lens-node/docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh

# Expose default ports
EXPOSE 9501 3000

# Set entrypoint
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]

# Default command (will be passed to entrypoint which adds node dist/cli/bin.js)
CMD ["run"]
26 changes: 26 additions & 0 deletions Dockerfile.local
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
FROM node:22-trixie-slim

WORKDIR /app

# Enable corepack and install pnpm
ENV PNPM_HOME=/root/.local/share/pnpm
ENV PATH=$PNPM_HOME:$PATH
RUN corepack enable && corepack prepare pnpm@latest --activate

# Install build dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
python3 \
make \
g++ \
&& rm -rf /var/lib/apt/lists/*

# Copy package files
COPY package.json pnpm-lock.yaml* ./

# Install dependencies
RUN pnpm install --frozen-lockfile

# Copy built dist
COPY dist ./dist

ENTRYPOINT ["node", "dist/cli/bin.js"]
46 changes: 46 additions & 0 deletions Dockerfile.simple
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
# Simple single-stage Dockerfile for lens-node
FROM node:18-slim

WORKDIR /app

# Install pnpm and build tools
RUN npm install -g pnpm && \
apt-get update && \
apt-get install -y python3 make g++ && \
rm -rf /var/lib/apt/lists/*

# Copy lens-sdk source
COPY ./lens-sdk /lens-sdk
WORKDIR /lens-sdk
RUN pnpm install --frozen-lockfile && pnpm build

# Copy lens-node source
WORKDIR /app
COPY ./lens-node/package.json ./lens-node/pnpm-lock.yaml ./

# Install dependencies with local lens-sdk
RUN sed -i 's|"@riffcc/lens-sdk": ".*"|"@riffcc/lens-sdk": "file:/lens-sdk"|' package.json && \
pnpm install --no-frozen-lockfile

# Copy lens-node source and build
COPY ./lens-node/ ./
RUN pnpm build

# Create directory for lens-node data
RUN mkdir -p /root/.lens-node

# Make binary executable
RUN chmod +x dist/cli/bin.js

# Copy entrypoint script
COPY ./lens-node/docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh

# Expose default ports
EXPOSE 9501 3000

# Set entrypoint
ENTRYPOINT ["/usr/local/bin/docker-entrypoint.sh"]

# Default command (will be passed to entrypoint which adds node dist/cli/bin.js)
CMD ["run"]
46 changes: 46 additions & 0 deletions bin.js.bak
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
#!/usr/bin/env node
import yargs from 'yargs';
import { hideBin } from 'yargs/helpers';
import runCommand from './commands/run.js';
import setupCommand from './commands/setup.js';
import importCommand from './commands/import.js';
import { generateMigrationCommand } from './commands/generate-migration.js';
import { migrateCommand } from './commands/migrate.js';
import { undoCommand } from './commands/undo.js';
import { releasesMigrateCommand } from './commands/releases-migrate.js';
import { programMigrateCommand } from './commands/program-migrate.js';
import { readFileSync } from 'fs';
import { fileURLToPath } from 'url';
import path from 'path';
const __filename = fileURLToPath(import.meta.url);
const __dirname = path.dirname(__filename);
const packageJsonPath = path.resolve(__dirname, '../../package.json');
const packageJson = JSON.parse(readFileSync(packageJsonPath, 'utf-8'));
yargs(hideBin(process.argv))
.scriptName('lens-node')
.command(setupCommand)
.command(runCommand)
.command(importCommand)
.command(generateMigrationCommand)
.command(migrateCommand)
.command(undoCommand)
.command(releasesMigrateCommand)
.command(programMigrateCommand)
.demandCommand(1, 'A command must be specified.')
.strict()
.help()
.alias('h', 'help')
.version(packageJson.version)
.alias('v', 'version')
.fail((msg, err, yargsInstance) => {
if (err) {
console.error('Error:', err.message);
}
else if (msg) {
console.error(`Error: ${msg}\n`); // Then print the specific message
}
yargsInstance.showHelp(); // Show yargs generated help for context
process.exit(1);
})
.parse();
//# sourceMappingURL=bin.js.map
3 changes: 2 additions & 1 deletion docker-build.sh
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
docker build -t zorlin/lens-node:0.1.16 --build-arg TAG=0.1.16 .
docker build -t zorlin/lens-node:0.1.48-a13 --build-arg TAG=0.1.48-a13 .
docker push zorlin/lens-node:0.1.48-a13
Loading