Create Data_challenge

[yadavved01]

No description provided.

[yadavved01]

The purpose of this code is to calculate the percentage of "positive consent" given for each date in the events data.

Here's a summary of what the code does:

Initializes a Spark session and reads the events data from a JSON file into a DataFrame named "eventsDF".

Filters the events based on their type, so that only "pageview", "consent.given", and "consent.asked" events are kept.

Adds a new column "date" to the filtered events DataFrame by converting the "datetime" field to a date.

Extracts the "enabled purposes" field from the "user.token" field in the events data, and adds a new column "positive_consent" to the DataFrame indicating whether the user has given positive consent (based on the size of the "enabled_purposes" field).

Groups the events by "date" and "positive_consent", counting the number of events for each combination.

Pivots the grouped DataFrame to calculate the percentage of positive consent for each date.

Displays the result DataFrame and stops the Spark session.