Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 8 additions & 55 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,64 +1,17 @@
Assignment 4 - Creative Coding: Interactive Multimedia Experiences
===

For this assignment we will focus on client-side development using popular audio/graphics/visualization technologies. The goal of this assignment is to refine our JavaScript knowledge while exploring the multimedia capabilities of the browser.

- [Three.js Tutorial](https://github.com/jmcuneo/cs4241-guides/blob/master/using.three.md)
- [WebAudio / Canvas Tutorial](https://github.com/jmcuneo/cs4241-guides/blob/main/using.webaudio_and_canvas.md)
- [SVG + D3 tutorial](https://github.com/jmcuneo/cs4241-guides/blob/main/using.svg_and_d3.md)
## Web Audio Visualizer

Baseline Requirements
https://a4-colinlemire.onrender.com/
---

Your application is required to implement the following functionalities:
# Project Description

- A server created using Express. This server can be as simple as needed.
- A client-side interactive experience using at least one of the following web technologies frameworks.
- [Three.js](https://threejs.org/): A library for 3D graphics / VR experiences
- [D3.js](https://d3js.org): A library that is primarily used for interactive data visualizations
- [Canvas](https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API): A 2D raster drawing API included in all modern browsers
- [SVG](https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API): A 2D vector drawing framework that enables shapes to be defined via XML.
- [Web Audio API](https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API): An API for audio synthesis, analysis, processing, and file playback.
- A user interface for interaction with your project, which must expose at least four parameters for user control. [tweakpane](https://cocopon.github.io/tweakpane/) is highly recommended for this, but you can also use regular HTML `<input>` tags (the `range` type is useful to create sliders). You might also explore interaction by tracking mouse movement via the `window.onmousemove` event handler in tandem with the `event.clientX` and `event.clientY` properties. Consider using the [Pointer Events API](https://developer.mozilla.org/en-US/docs/Web/API/Pointer_events) to ensure that that both mouse and touch events will both be supported in your app.
- Your application should display basic documentation for the user interface when the application first loads.
This project is a web application which plays and visualizes an mp3 file using Canvas and Web Audio API. The website will load you to a page with a button to start the visualizer. The webpage will give you a warning as well, but when you click the button the audio will immediately start to play, so you may want to turn your volume down before hitting the button. Once the button is pressed, a black background will be loaded in on which the audio will be visualized. You will also be presented with 4 interactive objects below the image - a bar to control the time in the song and the audio level (turning the audio down will also reduce the size of the audio being visualized), a button to randomize the color of the visualization, a dropdown menu to change how the audio is visualized (bars, waveform, or radially), and two buttons to either increase or decrease the bar size in the visualization without affecting the audio level.

The interactive experience should possess a reasonable level of complexity. Some examples:
### Three.js
- A generative algorithm creates simple agents that move through a virtual world. Your interface controls the behavior / appearance of these agents.
- A simple 3D game. You really want this to be a simple as possible or it will be outside the scope of this assignment.
- An 3D audio visualization of a song of your choosing. User interaction should control aspects of the visualization.
### Canvas
- Implement a generative algorithm such as [Conway's Game of Life](https://bitstorm.org/gameoflife/) (or 1D cellular automata) and provide interactive controls. Note that the Game of Life has been created by 100s of people using `<canvas>`; we'll be checking to ensure that your implementation is not a copy of these.
- Design a 2D audio visualizer of a song of your choosing. User interaction should control visual aspects of the experience.
### Web Audio API
- Create a screen-based musical instrument using the Web Audio API. You can use projects such as [Interface.js](http://charlie-roberts.com/interface/) or [Nexus UI](https://nexus-js.github.io/ui/api/#Piano) to provide common musical interface elements, or use dat.GUI in combination with mouse/touch events (use the Pointer Events API). Your GUI should enable users to control aspects of sound synthesis. If you want to use higher-level instruments instead of the raw WebAudio API sounds, consider trying the instruments provided by [Tone.js]() or [Gibber](https://github.com/charlieroberts/gibber.audio.lib).
### D3.js
- Create visualizations using the datasets found at [Awesome JSON Datasets](https://github.com/jdorfman/Awesome-JSON-Datasets). Experiment with providing different visualizations of the same data set, and providing users interactive control over visualization parameters and/or data filtering. Alternatively, create a single visualization with using one of the more complicated techniques shown at [d3js.org](d3js.org) and provide meaningful points of interaction for users.
My biggest challenge while making this application was definitely making the different types of visualization forms work. The different types of visualizations are very difficult to make, with a lot of math needed to figure out how to properly display the audio. I used the two websites below to help me with the different types of audio visualization. I followed the guide on the professors Github to make the default 'bars' variation

Deliverables
---

Do the following to complete this assignment:

1. Implement your project with the above requirements.
1. Test your project to make sure that when someone goes to your main page on Render, etc., it displays correctly.
1. Ensure that your project has the proper naming scheme `a4-firstname-lastname` so we can find it.
1. Fork this repository and modify the README to the specifications below.
1. Create and submit a Pull Request to the original repo. Name the pull request using the following template: `a4-firstname-lastname`.

Grading
---
Unlike previous assignments, this assignment will be solely graded on whether or not you successfully complete it. Partial credit will be generously given.

Sample Readme (delete the above when you're ready to submit, and modify the below so with your links and descriptions)
---

## Your Web Application Title

your hosting link e.g. http://a4-charlieroberts.glitch.me

Include a very brief summary of your project here. Images are encouraged when needed, along with concise, high-level text. Be sure to include the following:
While the website is easily navigable by itself, I think it is worth saying that if you use this code locally, all you have to do to change the song is swap the mp3 file being used. The audio is loaded in on line 50 in main.js - if you wish to change the song, feel free. I used many different songs before settling on 'Black Cow' by Steely Dan

- the goal of the application
- challenges you faced in realizing the application
- the instructions you present in the website should be clear enough to use the application, but if you feel any need to provide additional instructions please do so here.
https://www.hollyland.com/blog/tips/make-audio-visualizer - Radial
https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API/Visualizations_with_Web_Audio_API - Waveform
24 changes: 24 additions & 0 deletions package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
{
"name": "a4-colinlemire",
"version": "1.0.0",
"description": "Assignment 4 - Creative Coding: Interactive Multimedia Experiences\r ===",
"main": "server.js",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1",
"start": "node server.js"
},
"repository": {
"type": "git",
"url": "git+https://github.com/colinl13/a4-ColinLemire.git"
},
"keywords": [],
"author": "",
"license": "ISC",
"bugs": {
"url": "https://github.com/colinl13/a4-ColinLemire/issues"
},
"homepage": "https://github.com/colinl13/a4-ColinLemire#readme",
"dependencies": {
"express": "^5.1.0"
}
}
Binary file added public/black_cow.mp3
Binary file not shown.
49 changes: 49 additions & 0 deletions public/index.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
<!doctype html>
<html lang='en'>
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>Web Audio Visualizer</title>
<link href="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/css/bootstrap.min.css" rel="stylesheet">
</head>
<body class="bg-dark text-light">
<div class="container-fluid">
<div class="row justify-content-center">
<div class="col-12 col-md-10 col-lg-8">
<div class="text-center py-5">
<h1 id="header" class="display-4 mb-4">Web Audio + Canvas Visualizer</h1>
<h1 id="header2" class="display-5 mb-4" style="display:none;">Black Cow - Steely Dan</h1>
<div class="alert alert-warning" role="alert" id="warning">
<p id="warning" class="mb-0">Warning - Starting the Visualizer will also automatically start the audio</p>
</div>
<button class="btn btn-primary btn-lg mb-4">Start Visualizer</button>
</div>

<div id="audio-visualizer" class="text-center">
<script src="main.js">
// Load audio visualizer
</script>
</div>

<!-- Randomize Color Button -->
<div class="text-center mt-4">
<button class="btn btn-secondary" style="display:none;" id="color-randomizer">Randomize Color</button>
</div>
<br>
<!-- Visual Style Dropdown -->
<div class="d-flex justify-content-center" id="visual-style" style="display:none;"></div>

<!-- Bar Controls -->
<div class="bar-controls text-center mt-4 d-flex justify-content-center">
<div class="btn-group" role="group" id="bar-controls" style="display:none;">
<button class="btn btn-outline-light" id="decrease-bars">-</button>
<div class="btn btn-outline-light" id="bar-count">Bars: </div>
<button class="btn btn-outline-light" id="increase-bars">+</button>
</div>
</div>
</div>
</div>
</div>
<script src="https://cdn.jsdelivr.net/npm/bootstrap@5.3.0/dist/js/bootstrap.bundle.min.js"></script>
</body>
</html>
166 changes: 166 additions & 0 deletions public/main.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,166 @@
const start = function() {
// Hide the initial start button after click
const startButton = document.querySelector('button');
startButton.style.display = 'none';
const header = document.getElementById('header');
header.style.display = 'none';
const warning = document.getElementById('warning');
warning.style.display = 'none';
// Show the name of the song and the artist who wrote it
const header2 = document.getElementById('header2');
header2.style.display = 'inline';



const canvas = document.createElement('canvas');
document.getElementById("audio-visualizer").appendChild(canvas);
var br = document.createElement("br");
document.getElementById("audio-visualizer").appendChild(br);
canvas.width = canvas.height = 512;
const ctx = canvas.getContext('2d');

// Color state for rectangles; default is 'lime'
let rectColor = 'lime';

// Visualization mode UI
const visualStyleHost = document.getElementById('visual-style');
visualStyleHost.style.display = 'inline';
let mode = 'bars'; // mode set to bars by default
const modeSelect = document.createElement('select');
// Options map to algorithms implemented below in draw()
[
{ value: 'bars', label: 'Bars' },
{ value: 'waveform', label: 'Waveform' },
{ value: 'radial', label: 'Radial' }
].forEach(function(menu) {
const option = document.createElement('option');
option.value = menu.value;
option.textContent = menu.label;
modeSelect.appendChild(option);
});
modeSelect.value = mode;
modeSelect.addEventListener('change', function(e) {
mode = e.target.value;
});

visualStyleHost.appendChild(modeSelect);


// Create audio element with black_cow.mp3
const audioElement = new Audio('black_cow.mp3');
audioElement.controls = true;
document.getElementById("audio-visualizer").appendChild(audioElement);

// audio init
const audioCtx = new (window.AudioContext || window.webkitAudioContext)();
const source = audioCtx.createMediaElementSource(audioElement);

const analyser = audioCtx.createAnalyser();
analyser.fftSize = 1024;

source.connect(analyser);
analyser.connect(audioCtx.destination);

const results = new Uint8Array(analyser.frequencyBinCount);
// Used for Waveform vizualization
const timeData = new Uint8Array(analyser.fftSize);

// Button to randomize color
const colorButton = document.getElementById('color-randomizer');
colorButton.style.display = 'inline';
colorButton.addEventListener('click', function() {
rectColor = getRandomColor();
});

// Buttons to increase/decrease bar size
const barControls = document.getElementById('bar-controls');
barControls.style.display = 'inline';
var barCountDisplay = document.getElementById('bar-count');
let barSize = 2.0; // Initial bar width
const increaseBarsButton = document.getElementById('increase-bars');
increaseBarsButton.onclick = function() {
if (barSize < 5.0) {
barSize += .25;
barCountDisplay.textContent = `Visual Size: ${barSize}`;
// console.log("barSize is " + barSize);
}
};
const decreaseBarsButton = document.getElementById('decrease-bars');
decreaseBarsButton.onclick = function() {
if (barSize >= 0.25) {
barSize -= .25;
barCountDisplay.textContent = `Visual Size: ${barSize}`;
// console.log("barSize is " + barSize);
}
};
barCountDisplay.textContent = `Visual Size: ${barSize}`;


// Helper function to generate random color
function getRandomColor() {
var letters = '0123456789ABCDEF';
var color = '#';

for (var i = 0; i < 6; i++) {
color += letters[Math.floor(Math.random() * 16)];
}

return color;
}

function draw() {
window.requestAnimationFrame(draw);
// Set black background
ctx.fillStyle = 'black';
ctx.fillRect(0, 0, canvas.width, canvas.height);

if (mode === 'bars') {
analyser.getByteFrequencyData(results);
ctx.fillStyle = rectColor;
for (let i = 0; i < analyser.frequencyBinCount; i++) {
ctx.fillRect(i*1.5, canvas.height - (barSize*results[i]), 1.5, barSize*results[i]);
}
} else if (mode === 'waveform') {
analyser.getByteTimeDomainData(timeData);
ctx.strokeStyle = rectColor;
ctx.lineWidth = 3;
ctx.beginPath();
const sliceWidth = canvas.width / timeData.length;
let x = 0;
for (let i = 0; i < timeData.length; i++) {
const v = timeData[i] / 128; // Bytes go from 0 to 255, middle is 128
const y = (v * canvas.height) / 2;
if (i === 0)
ctx.moveTo(x, y);
else
ctx.lineTo(x, y);
x += sliceWidth;
}
ctx.stroke();
} else if (mode === 'radial') { // Personal favorite
analyser.getByteFrequencyData(results);
const cx = canvas.width / 2;
const cy = canvas.height / 2;
const maxRadius = Math.min(cx, cy) - 4;
ctx.strokeStyle = rectColor;
ctx.lineWidth = 1;
for (let i = 0; i < analyser.frequencyBinCount; i++) {
const angle = (i / analyser.frequencyBinCount) * Math.PI * (1.52*barSize);
const magnitude = results[i] / 255;
const r = magnitude * maxRadius;
const x = cx + Math.cos(angle) * r;
const y = cy + Math.sin(angle) * r;
ctx.beginPath();
ctx.moveTo(cx, cy);
ctx.lineTo(x, y);
ctx.stroke();
}
}
}
draw();

audioElement.play();
};
window.onload = () => document.querySelector('button').onclick = start;


16 changes: 16 additions & 0 deletions server.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
const express = require('express');
const path = require('path');
const app = express();
const PORT = process.env.PORT || 3000;

// Serve static files from the public directory
app.use(express.static(path.join(__dirname, 'public')));

// Default route
app.get('/', (req, res) => {
res.sendFile(path.join(__dirname, 'public', 'index.html'));
});

app.listen(PORT, () => {
console.log(`Server running on http://localhost:${PORT}`);
});