Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 25 additions & 3 deletions AI_ASSIST.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,38 @@ Document one session where you used an LLM to help with a query or a design deci

## The problem

TODO: What were you trying to solve? Paste the relevant SQL or schema fragment.
My database was working but the next day it said connection is broken
Connection to hyf-data-pg.postgres.database.azure.com:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
Connection refused: getsockopt

## The prompt

TODO: What did you ask the AI? Include the context you provided.
why is my connection to my database broken with error Connection refused: getsockopt

## The response

TODO: What did it suggest? Did it work first try?
refused: getsockopt

"Connection refused: getsockopt" is a very common socket-level error. It means your client tried to open a TCP connection to the database but the target machine actively refused it (or nothing was listening).Most Common Causes (in order)Rank
Cause
Why it happens
1
Database server is not running
Service crashed, wasn't started, or was stopped
2
Wrong host/port
You're connecting to localhost:5432 but the DB is on another machine or different port
3
DB only listening on localhost
Many DBs (especially Postgres, MySQL) by default bind to 127.0.0.1 only
4
Firewall
iptables, ufw, firewalld, cloud security groups, etc. blocking the port

IT DIDNT WORK

## Reflection

TODO: Did you understand *why* the suggestion worked, or did you accept it blindly?

i did not understand ,so i requested help and used a localhost postgres instead of the azure postgres
Binary file added assets/borough_count.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 8 additions & 8 deletions data_dictionary.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ Document both views. State the grain in one sentence, identify the keys, and lis

## vw_fact_trips

- **Grain:** TODO (one sentence, e.g. "One row per ...")
- **Primary key:** TODO
- **Foreign keys:** TODO
- **Measures:** TODO (columns you would SUM or AVG)
- **Grain:** one row per taxi trip that has fare_amount >=0
- **Primary key:** vendor_id
- **Foreign keys:** pickup_location_id, dropoff_location_id
- **Measures:** fare_amount,trip_distance,tip_amount,passenger_count

## vw_dim_zones

- **Grain:** TODO
- **Primary key:** TODO
- **Foreign keys:** TODO (or "none")
- **Measures:** TODO (or "none, descriptive attributes only")
- **Grain:** one zone per day
- **Primary key:** location id
- **Foreign keys:** ( "none")
- **Measures:** "none, descriptive attributes only"
28 changes: 23 additions & 5 deletions schema_setup.sql
Original file line number Diff line number Diff line change
Expand Up @@ -5,8 +5,12 @@
-- TODO: complete the SELECT (location_id, zone, borough).
CREATE OR REPLACE VIEW vw_dim_zones AS
SELECT
-- TODO
FROM nyc_taxi.raw_zones;
rz.location_id,
rz.zone,
rz.borough
FROM nyc_taxi.raw_zones ;

;

-- Fact: one row per taxi trip.
-- - Exclude rows where fare_amount is less than 0.
Expand All @@ -15,12 +19,26 @@ FROM nyc_taxi.raw_zones;
-- TODO: complete the SELECT and the WHERE.
CREATE OR REPLACE VIEW vw_fact_trips AS
SELECT
-- TODO
vendor_id,
pickup_datetime::timestamp AS pickup_datetime,
dropoff_datetime,
passenger_count,
pickup_location_id,
dropoff_location_id,
trip_distance,
fare_amount,
tip_amount,
payment_type
FROM nyc_taxi.raw_trips
-- TODO: WHERE fare_amount >= 0
;
WHERE fare_amount >= 0;

-- Join-readiness test (run after creating the views; it must run without error
-- and return a count close to the vw_fact_trips row count):
-- SELECT COUNT(*) FROM vw_fact_trips f
-- JOIN vw_dim_zones d ON f.pickup_location_id = d.location_id;

select * from vw_fact_trips f
join vw_dim_zones d on f.pickup_location_id = d.location_id;



23 changes: 23 additions & 0 deletions validation_queries.sql
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,38 @@
-- 1. Duplicate check: are there rows with the same vendor_id, pickup_datetime, dropoff_datetime?
-- TODO: GROUP BY the three columns and keep only groups with HAVING COUNT(*) > 1.

SELECT vendor_id, pickup_datetime, dropoff_datetime, COUNT(*) AS duplicate_count
FROM nyc_taxi.raw_trips
GROUP BY vendor_id, pickup_datetime, dropoff_datetime
HAVING COUNT(*) > 1
ORDER BY duplicate_count DESC;


-- 2. Null integrity: how many rows have a NULL pickup_location_id or dropoff_location_id?
-- TODO: count the NULLs (COUNT(*) FILTER (WHERE ... IS NULL) is handy for several columns at once).

SELECT
COUNT(*) AS total_rows,
COUNT(*) FILTER (WHERE pickup_location_id IS NULL) AS null_pickup_location_id,
COUNT(*) FILTER (WHERE dropoff_location_id IS NULL) AS null_dropoff_location_id
FROM nyc_taxi.raw_trips;


-- 3. Range validation: what are the min and max fare_amount? Are there negative values?
-- TODO: SELECT MIN(fare_amount), MAX(fare_amount), and a count of rows where fare_amount < 0.

SELECT
MIN(fare_amount) AS min_fare_amount,
MAX(fare_amount) AS max_fare_amount,
COUNT(*) FILTER (WHERE fare_amount < 0) AS negative_fare_count
FROM nyc_taxi.raw_trips;


-- 4. Relationship check: which pickup_location_id values in nyc_taxi.raw_trips do NOT exist in nyc_taxi.raw_zones?
-- TODO: LEFT JOIN nyc_taxi.raw_zones ... WHERE z.location_id IS NULL (or NOT EXISTS).
-- Do NOT use NOT IN: a single NULL in the subquery hides every orphan.

SELECT t.pickup_location_id
FROM nyc_taxi.raw_trips t
LEFT JOIN nyc_taxi.raw_zones z ON t.pickup_location_id = z.location_id
WHERE z.location_id IS NULL
43 changes: 42 additions & 1 deletion verification_results.sql
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,61 @@
-- Borough and zone names live in vw_dim_zones, so join on pickup_location_id = location_id.

-- 1. Volume: how many total rows in vw_fact_trips? How many rows per borough?

SELECT COUNT(*) AS total_rows FROM vw_fact_trips;


-- What is the most common pickup/dropoff location combination?

Select d.borough, COUNT(*) AS rows_per_borough
from vw_fact_trips f
join vw_dim_zones d on f.pickup_location_id = d.location_id
group by d.borough
order by rows_per_borough desc;


-- TODO
-- (Take a screenshot of the per-borough counts and save it as assets/borough_count.png.)



-- 2. Revenue: which pickup zone (name, not ID) generated the highest total fare_amount?

SELECT d.zone, SUM(f.fare_amount) AS total_fare
FROM vw_fact_trips f
JOIN vw_dim_zones d ON f.pickup_location_id = d.location_id
GROUP BY d.zone
ORDER BY total_fare DESC
LIMIT 1;
-- Which pickup zone collected the highest total fare_amount on any single day?
-- TODO
SELECT d.zone, f.pickup_datetime::date AS pickup_date, SUM(f.fare_amount) AS total_fare
FROM vw_fact_trips f
JOIN vw_dim_zones d ON f.pickup_location_id = d.location_id
GROUP BY d.zone, pickup_date
ORDER BY total_fare DESC
LIMIT 1;


-- 3. Geospatial: total number of trips and average trip_distance for each borough.
-- TODO

SELECT d.borough, COUNT(*) AS total_trips, AVG(f.trip_distance) AS avg_trip_distance
FROM vw_fact_trips f
JOIN vw_dim_zones d ON f.pickup_location_id = d.location_id
GROUP BY d.borough
ORDER BY total_trips DESC;

-- 4. Time patterns: which day of the week had the highest total tip_amount?

SELECT EXTRACT(DOW FROM f.pickup_datetime) AS day_of_week, SUM(f.tip_amount) AS total_tip
FROM vw_fact_trips f
GROUP BY day_of_week
ORDER BY total_tip DESC
LIMIT 1;
-- What hour of the day has the highest average tip?
-- TODO
SELECT EXTRACT(HOUR FROM f.pickup_datetime) AS hour_of_day, AVG(f.tip_amount) AS avg_tip
FROM vw_fact_trips f
GROUP BY hour_of_day
ORDER BY avg_tip DESC
LIMIT 1;