DS504/CS586 - Big Data Analytics - Spring 2025

This repository contains projects, assignments, and notes from the DS504/CS586—Big Data Analytics course at Worcester Polytechnic Institute (WPI), which Prof. Yanhua Li instructed during the Spring 2020 semester.

📌 Course Overview

The course focuses on computational techniques and algorithms for measuring, analyzing, and mining patterns in large-scale datasets. It covers key big data concepts and real-world applications in areas such as urban computing, social media analysis, and recommender systems.

🔍 Topics Covered

• Large-scale data sampling and estimation
• Data cleaning and management
• Clustering techniques and pattern recognition
• Advanced machine learning techniques for big data
• Applications in urban computing, social media, and recommender systems

📂 Repository Contents

This repository includes:

• 📁 Assignments: Problem-solving exercises and programming tasks.
• 📁 Projects: Hands-on projects implementing big data analytics techniques.
• 📁 Notes: Lecture summaries and important takeaways.

🧑‍🏫 Course Instructor

Prof. Yanhua Li

• Office: AK130
• Email: yli15@wpi.edu
• Website: http://wpi.edu/~yli15/

🎓 Teaching Assistants

• Guojun Wu (gwu@wpi.edu)
• Huimin Ren (hren@wpi.edu)

📅 Office Hours

• Prof. Yanhua Li:

  • Tuesdays & Thursdays: 10:00 AM - 11:00 AM
  • Additional by appointment

• Teaching Assistants:

  • Wednesdays & Thursdays: 2:00 PM - 3:00 PM
  • Additional by appointment

🛠 Technologies Used

• Python (Pandas, NumPy, Scikit-Learn, TensorFlow, PyTorch)
• SQL for data management
• Hadoop & Spark for large-scale data processing
• Jupyter Notebooks for analysis and visualization