Skip to content

concurrent emailing #12

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
djdiptayan1 wants to merge 8 commits into from
Closed

concurrent emailing #12

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
82ea19e
Merge pull request #6 from djdiptayan1/development
djdiptayan1 Jul 22, 2025
2bf4cd7
Merge pull request #7 from djdiptayan1/development
djdiptayan1 Jul 25, 2025
4a30826
Merge pull request #8 from djdiptayan1/development
djdiptayan1 Aug 16, 2025
192e778
Merge pull request #9 from djdiptayan1/development
djdiptayan1 Aug 20, 2025
c14725a
Merge pull request #10 from djdiptayan1/development
djdiptayan1 Sep 6, 2025
daa7bdc
Merge pull request #11 from djdiptayan1/development
djdiptayan1 Sep 9, 2025
9a28261
concurrent emailing
djdiptayan1 Sep 17, 2025
62386ad
templates
djdiptayan1 Oct 13, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
243 changes: 156 additions & 87 deletions beacon/app/api/SendEmails/route.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { SESClient, SendEmailCommand } from '@aws-sdk/client-ses';
import * as XLSX from 'xlsx';
import { promises as fs } from 'fs';
import path from 'path';
import pLimit from 'p-limit';
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

🧩 Analysis chain

Confirm p-limit is declared and compatible with ESM/Next runtime.

Ensure dependency is present (v4+ is ESM) and not duplicated via subdeps.

🏁 Script executed:

#!/bin/bash
# Verify dependency and version
rg -n '"p-limit"' package.json
jq -r '.dependencies["p-limit"] // .devDependencies["p-limit"]' package.json

Length of output: 123

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "PWD: $(pwd)"
echo
echo "---- find package.json files ----"
find . -name package.json -print || true

echo
echo "---- search for occurrences of 'p-limit' ----"
rg -n --hidden -S "p-limit" || true

echo
echo "---- show beacon/app/api/SendEmails/route.js (if present) ----"
if [ -f "beacon/app/api/SendEmails/route.js" ]; then
  sed -n '1,240p' beacon/app/api/SendEmails/route.js
else
  echo "beacon/app/api/SendEmails/route.js not found"
fi

echo
echo "---- inspect any package.json found for p-limit ----"
for f in $(find . -name package.json -print); do
  echo "---- $f ----"
  if command -v jq >/dev/null 2>&1; then
    jq -r '.name // "<no name>" as $n | ($n + " -> p-limit: " + (.dependencies["p-limit"] // .devDependencies["p-limit"] // "<not declared>"))' "$f" || cat "$f"
  else
    grep -n '"p-limit"' "$f" || echo "<jq missing; grep ran>"
  fi
done

echo
echo "---- search common lockfiles for p-limit versions ----"
rg -n --hidden -S "p-limit" package-lock.json yarn.lock pnpm-lock.yaml pnpm-lock.json || true

Length of output: 10533

🏁 Script executed:

#!/bin/bash
set -euo pipefail

echo "PWD: $(pwd)"
echo

echo "---- beacon/package.json (full) ----"
if [ -f "beacon/package.json" ]; then
  sed -n '1,240p' beacon/package.json
else
  echo "beacon/package.json not found"
fi

echo
echo "---- beacon/package-lock.json: p-limit matches (show surrounding lines) ----"
if [ -f "beacon/package-lock.json" ]; then
  rg -n --hidden -S "p-limit" beacon/package-lock.json -n -C3 || true
  echo
  echo "---- show node_modules/p-limit block if present ----"
  rg -n --hidden -S '"node_modules/p-limit":' -n -C6 beacon/package-lock.json || true
else
  echo "beacon/package-lock.json not found"
fi

echo
echo "---- beacon/app/api/SendEmails/route.js (search for runtime/export) ----"
if [ -f "beacon/app/api/SendEmails/route.js" ]; then
  rg -n "export const runtime|export const config|export const dynamic" beacon/app/api/SendEmails/route.js || echo "no explicit runtime/config exports found in route.js"
  echo
  sed -n '1,400p' beacon/app/api/SendEmails/route.js
else
  echo "beacon/app/api/SendEmails/route.js not found"
fi

Length of output: 16037

Declare p-limit in package.json and choose an ESM-compatible version

  • beacon/app/api/SendEmails/route.js imports p-limit (line 6) but beacon/package.json does not declare it; package-lock shows node_modules/p-limit@3.1.0 as a transitive dev-only dependency (risk: omitted in production installs).
  • p-limit v3 is CommonJS; v4+ is ESM-only. Action: add p-limit to beacon/package.json dependencies and pin the major you need — use ^4.x if this route may run in Edge/ESM, or ^3.x if you must remain CJS — then run install to update the lockfile.
🤖 Prompt for AI Agents 
In beacon/app/api/SendEmails/route.js around line 6, p-limit is imported but not
declared in beacon/package.json and the transitive node_modules/p-limit@3.1.0 is
a dev-only dep; add p-limit to beacon/package.json "dependencies" (not
devDependencies) and pin the major you need: use "^4.x" if this route must run
in ESM/Edge environments or stay with "^3.x" if you must remain CommonJS, then
run npm/yarn install to update the lockfile and ensure it’s installed in
production installs.


export async function POST(request) {
// No tempDir needed; use in-memory buffers
Expand Down Expand Up @@ -112,94 +113,162 @@ export async function POST(request) {
const results = [];
let processedCount = 0;

for (const row of data) {
try {
processedCount++;
console.log(`Processing email ${processedCount}/${data.length} for ${row.Email}`);

// Skip rows with missing email or name
if (!row.Email || !row.Name) {
results.push({
email: row.Email || 'Unknown',
name: row.Name || 'Unknown',
status: 'skipped',
message: 'Missing email or name'
});
continue;
}

// Replace placeholders in template
let personalizedTemplate = template;

// Replace {{Recipient_name}} with the actual name
personalizedTemplate = personalizedTemplate.replace(/{{Recipient_name}}/g, row.Name);

// Replace any other placeholders that might exist
Object.keys(row).forEach(key => {
const placeholder = new RegExp(`{{${key}}}`, 'g');
personalizedTemplate = personalizedTemplate.replace(placeholder, row[key]);
});

const isPlainTextOnly = false; // We always send as HTML now since rich text editor produces HTML

// Create SES email parameters
const emailParams = {
Source: senderEmail,
Destination: {
ToAddresses: [row.Email],
},
Message: {
Subject: {
Data: subject,
Charset: 'UTF-8',
},
Body: {
Html: {
Data: personalizedTemplate,
Charset: 'UTF-8',
// Instead of for...of
const limit = pLimit(12); // 12 concurrent sends (SES default rate)

// for (const row of data) {
// try {
// processedCount++;
// console.log(`Processing email ${processedCount}/${data.length} for ${row.Email}`);

// // Skip rows with missing email or name
// if (!row.Email || !row.Name) {
// results.push({
// email: row.Email || 'Unknown',
// name: row.Name || 'Unknown',
// status: 'skipped',
// message: 'Missing email or name'
// });
// continue;
// }

// // Replace placeholders in template
// let personalizedTemplate = template;

// // Replace {{Recipient_name}} with the actual name
// personalizedTemplate = personalizedTemplate.replace(/{{Recipient_name}}/g, row.Name);

// // Replace any other placeholders that might exist
// Object.keys(row).forEach(key => {
// const placeholder = new RegExp(`{{${key}}}`, 'g');
// personalizedTemplate = personalizedTemplate.replace(placeholder, row[key]);
// });

// const isPlainTextOnly = false; // We always send as HTML now since rich text editor produces HTML

// // Create SES email parameters
// const emailParams = {
// Source: senderEmail,
// Destination: {
// ToAddresses: [row.Email],
// },
// Message: {
// Subject: {
// Data: subject,
// Charset: 'UTF-8',
// },
// Body: {
// Html: {
// Data: personalizedTemplate,
// Charset: 'UTF-8',
// },
// },
// },
// };

// // Send email
// const command = new SendEmailCommand(emailParams);
// const response = await sesClient.send(command);

// results.push({
// email: row.Email,
// name: row.Name,
// status: 'success',
// message: 'Email sent successfully',
// messageId: response.MessageId
// });

// // Add a small delay between emails to avoid rate limiting
// if (processedCount < data.length) {
// await new Promise(resolve => setTimeout(resolve, 100));
// }

// } catch (error) {
// console.error(`Error sending email to ${row.Email}:`, error);

// // Provide specific error messages
// let errorMessage = error.message || 'Unknown error occurred';
// if (error.message.includes('Email address is not verified')) {
// errorMessage = 'Email address not verified in AWS SES';
// } else if (error.message.includes('AccessDenied')) {
// errorMessage = 'AWS SES permission denied - check IAM permissions';
// } else if (error.message.includes('InvalidParameterValue')) {
// errorMessage = 'Invalid email format or parameters';
// }

// results.push({
// email: row.Email,
// name: row.Name,
// status: 'error',
// message: errorMessage
// });
// }
// }
Comment on lines +119 to +206
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion

Remove commented‑out legacy loop.

Large commented blocks add noise and risk divergence.

Delete this block entirely from the PR.

🤖 Prompt for AI Agents 
In beacon/app/api/SendEmails/route.js around lines 119 to 206, remove the large
commented-out legacy for-loop block (the entire commented section that processes
rows, sends SES emails, and handles errors) to eliminate dead code and noise;
ensure no dependent references remain elsewhere in the file and run
tests/linting to verify nothing breaks after deletion.


Comment on lines +186 to +207
Copy link

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Large blocks of commented-out code should be removed instead of left in the codebase. This creates clutter and makes the code harder to maintain. The old implementation can be retrieved from version control if needed.

Suggested change
// } catch (error) {
// console.error(`Error sending email to ${row.Email}:`, error);
// // Provide specific error messages
// let errorMessage = error.message || 'Unknown error occurred';
// if (error.message.includes('Email address is not verified')) {
// errorMessage = 'Email address not verified in AWS SES';
// } else if (error.message.includes('AccessDenied')) {
// errorMessage = 'AWS SES permission denied - check IAM permissions';
// } else if (error.message.includes('InvalidParameterValue')) {
// errorMessage = 'Invalid email format or parameters';
// }
// results.push({
// email: row.Email,
// name: row.Name,
// status: 'error',
// message: errorMessage
// });
// }
// }

Copilot uses AI. Check for mistakes.
await Promise.all(
data.map((row, idx) =>
limit(async () => {
try {
processedCount++;
console.log(`Processing email ${processedCount}/${data.length} for ${row.Email}`);
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Don’t log full recipient emails (PII). Mask before logging.

Avoid emitting raw addresses to logs.

-                        console.log(`Processing email ${processedCount}/${data.length} for ${row.Email}`);
+                        const maskedEmail = typeof row.Email === 'string'
+                          ? row.Email.replace(/(.{2}).+(@.+)/, '$1***$2')
+                          : 'Unknown';
+                        console.log(`Processing email ${processedCount}/${data.length} for ${maskedEmail}`);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
console.log(`Processing email ${processedCount}/${data.length} for ${row.Email}`);
const maskedEmail = typeof row.Email === 'string'
? row.Email.replace(/(.{2}).+(@.+)/, '$1***$2')
: 'Unknown';
console.log(`Processing email ${processedCount}/${data.length} for ${maskedEmail}`);
🤖 Prompt for AI Agents 
In beacon/app/api/SendEmails/route.js around line 213 the code logs full
recipient emails (PII) with console.log(`Processing email
${processedCount}/${data.length} for ${row.Email}`); — replace this with a
masked form of the email before logging (e.g., keep first character and domain,
replace intervening characters with asterisks or show only the domain and a
short hashed/tokenized prefix), and ensure no other logs emit the raw row.Email;
update the logging statement to use the masked value and add a small helper or
inline logic to perform the masking consistently.

Comment on lines +212 to +213
Copy link

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The processedCount variable is being incremented concurrently without synchronization, which can lead to race conditions and inaccurate counts. Consider using an atomic counter or removing this logging since the concurrent processing makes the count unreliable for progress tracking.

Suggested change
processedCount++;
console.log(`Processing email ${processedCount}/${data.length} for ${row.Email}`);

Copilot uses AI. Check for mistakes.

if (!row.Email || !row.Name) {
results.push({
email: row.Email || 'Unknown',
name: row.Name || 'Unknown',
status: 'skipped',
message: 'Missing email or name'
});
return;
}

// Replace placeholders
let personalizedTemplate = template.replace(/{{Recipient_name}}/g, row.Name);
Object.keys(row).forEach(key => {
const placeholder = new RegExp(`{{${key}}}`, 'g');
personalizedTemplate = personalizedTemplate.replace(placeholder, row[key]);
});

Comment on lines +225 to +231
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Escape placeholder keys and avoid inserting "undefined" into the template.

Unescaped regex keys can mis‑match; undefined values render as the string "undefined".

-                        let personalizedTemplate = template.replace(/{{Recipient_name}}/g, row.Name);
-                        Object.keys(row).forEach(key => {
-                            const placeholder = new RegExp(`{{${key}}}`, 'g');
-                            personalizedTemplate = personalizedTemplate.replace(placeholder, row[key]);
-                        });
+                        const safe = v => (v == null ? '' : String(v));
+                        let personalizedTemplate = template.replace(/{{Recipient_name}}/g, safe(row.Name));
+                        for (const [key, value] of Object.entries(row)) {
+                            const escaped = key.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
+                            personalizedTemplate = personalizedTemplate
+                                .replace(new RegExp(`{{${escaped}}}`, 'g'), safe(value));
+                        }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Replace placeholders
let personalizedTemplate = template.replace(/{{Recipient_name}}/g, row.Name);
Object.keys(row).forEach(key => {
const placeholder = new RegExp(`{{${key}}}`, 'g');
personalizedTemplate = personalizedTemplate.replace(placeholder, row[key]);
});
// Replace placeholders
const safe = v => (v == null ? '' : String(v));
let personalizedTemplate = template.replace(/{{Recipient_name}}/g, safe(row.Name));
for (const [key, value] of Object.entries(row)) {
const escaped = key.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
personalizedTemplate = personalizedTemplate
.replace(new RegExp(`{{${escaped}}}`, 'g'), safe(value));
}
🤖 Prompt for AI Agents 
In beacon/app/api/SendEmails/route.js around lines 225 to 231, the loop builds a
RegExp from object keys without escaping special characters and blindly inserts
row[key] which can become the string "undefined"; update it to escape regex
metacharacters in the key before constructing the RegExp (so keys like "a.b" or
"name?" match literally) and only replace when row[key] is not null/undefined
(use an empty string or a safe default for falsy values), converting non-string
values to string explicitly before replacement.

const emailParams = {
Source: senderEmail,
Destination: { ToAddresses: [row.Email] },
Message: {
Subject: { Data: subject, Charset: 'UTF-8' },
Body: { Html: { Data: personalizedTemplate, Charset: 'UTF-8' } }
},
},
},
};

// Send email
const command = new SendEmailCommand(emailParams);
const response = await sesClient.send(command);

results.push({
email: row.Email,
name: row.Name,
status: 'success',
message: 'Email sent successfully',
messageId: response.MessageId
});

// Add a small delay between emails to avoid rate limiting
if (processedCount < data.length) {
await new Promise(resolve => setTimeout(resolve, 100));
}

} catch (error) {
console.error(`Error sending email to ${row.Email}:`, error);

// Provide specific error messages
let errorMessage = error.message || 'Unknown error occurred';
if (error.message.includes('Email address is not verified')) {
errorMessage = 'Email address not verified in AWS SES';
} else if (error.message.includes('AccessDenied')) {
errorMessage = 'AWS SES permission denied - check IAM permissions';
} else if (error.message.includes('InvalidParameterValue')) {
errorMessage = 'Invalid email format or parameters';
}

results.push({
email: row.Email,
name: row.Name,
status: 'error',
message: errorMessage
});
}
}
};

const command = new SendEmailCommand(emailParams);
const response = await sesClient.send(command);

results.push({
email: row.Email,
name: row.Name,
status: 'success',
message: 'Email sent successfully',
messageId: response.MessageId
});

} catch (error) {
console.error(`Error sending email to ${row.Email}:`, error);
let errorMessage = error.message || 'Unknown error occurred';
if (error.message.includes('Email address is not verified')) {
errorMessage = 'Email address not verified in AWS SES';
} else if (error.message.includes('AccessDenied')) {
errorMessage = 'AWS SES permission denied - check IAM permissions';
} else if (error.message.includes('InvalidParameterValue')) {
errorMessage = 'Invalid email format or parameters';
}
results.push({
email: row.Email,
name: row.Name,
status: 'error',
message: error.message || 'Unknown error occurred'
Copy link

Copilot AI Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The errorMessage variable that was constructed with specific error handling logic is being ignored. Line 266 should use errorMessage instead of error.message || 'Unknown error occurred' to preserve the enhanced error messages for AWS SES issues.

Suggested change
message: error.message || 'Unknown error occurred'
message: errorMessage

Copilot uses AI. Check for mistakes.
});
Comment on lines +252 to +267
Copy link
Copy Markdown

@coderabbitai coderabbitai bot Sep 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Use the mapped errorMessage and guard when error.message is undefined; also normalize matching.

Current code computes errorMessage but returns raw error.message and can throw on .includes when message is absent.

-                        console.error(`Error sending email to ${row.Email}:`, error);
-                        let errorMessage = error.message || 'Unknown error occurred';
-                        if (error.message.includes('Email address is not verified')) {
+                        console.error(`Error sending email to ${row.Email}:`, error);
+                        const rawMsg = String(error?.message ?? '');
+                        const msg = rawMsg.toLowerCase();
+                        let errorMessage = rawMsg || 'Unknown error occurred';
+                        if (msg.includes('not verified')) {
                             errorMessage = 'Email address not verified in AWS SES';
-                        } else if (error.message.includes('AccessDenied')) {
+                        } else if (msg.includes('accessdenied')) {
                             errorMessage = 'AWS SES permission denied - check IAM permissions';
-                        } else if (error.message.includes('InvalidParameterValue')) {
+                        } else if (msg.includes('invalidparametervalue')) {
                             errorMessage = 'Invalid email format or parameters';
+                        } else if (msg.includes('throttl')) {
+                            errorMessage = 'AWS SES rate limit exceeded - reduce concurrency or request a higher limit';
                         }
-                        results.push({
-                            email: row.Email,
-                            name: row.Name,
-                            status: 'error',
-                            message: error.message || 'Unknown error occurred'
-                        });
+                        results[idx] = {
+                            email: row.Email,
+                            name: row.Name,
+                            status: 'error',
+                            message: errorMessage
+                        };

Committable suggestion skipped: line range outside the PR's diff.

🤖 Prompt for AI Agents 
In beacon/app/api/SendEmails/route.js around lines 252-267, the code computes a
mapped errorMessage but still uses error.message and calls .includes on it which
can throw if undefined; fix by first extracting a safe string like const rawMsg
= (error && error.message) ? String(error.message) : '' , normalize it to
lowerCase for matching, run includes checks against the normalized string to set
errorMessage cases, then use the computed errorMessage when pushing results and
in the console.error (falling back to a generic message if empty).

}
})
)
);

// Calculate summary
const successCount = results.filter(r => r.status === 'success').length;
Expand Down
Binary file modified beacon/public/sample-data.xlsx
View file
Open in desktop
Binary file not shown.
Loading