Merge pull request #9 from isaacbernat/image_descriptions

Image descriptions

Author: isaacbernat

README.md

@@ -1,5 +1,5 @@
 # basepaint
-The main goal of this repository is to be a set of scripts to create an archive for the art generated by the [basepaint.xyz](https://basepaint.xyz/) community, including metadata and stats. This is the main development repository which only contains the code and instructions on how to run it. The actual archive and PDFs are available at the companion repository [basepaint-media](https://github.com/isaacbernat/basepaint-media)
+The main goal of this repository is to be a set of scripts to create an archive for the art generated by the [basepaint.xyz](https://basepaint.xyz/) community, including metadata, stats and descriptions. This is the main development repository which only contains the code and instructions on how to run it. The actual archive and PDFs are available at the companion repository [basepaint-media](https://github.com/isaacbernat/basepaint-media)
 
 ## Standard PDF edition
 The archive is available in PDF form there:
@@ -23,6 +23,59 @@ An extended PDF version is available, with twice as many pages as the Standard E
 - [Artworks with WIP from 401 to 500 in PDF](https://github.com/isaacbernat/basepaint-media/blob/main/pdf/basepaint_archive_0401_to_0500_with_WIP.pdf)
 - _"Artworks with WIP from 501"_ PDF and beyond are unavailable. It would exceed GitHub's git lfs quota for free repositories. If you are interested, you'll need to clone this repo (or preferably [basepaint-media](https://github.com/isaacbernat/basepaint-media)) and generate them on your own.
 
+## Descriptive PDF edition
+This edition uses one page per artwork, and one line per element with the following format `(X,Y) **Element**: Description`. X and Y represent the coordinates of the element within the image, with (0,0) being the top left corner and (100,100) the bottom right. X is the horizontal coordinate and Y the vertical one.
+
+The remaining blank space (in case there are not enough elements to fill the whole page) showcases a scaled down version of the original work with graded X and Y axis grids to help the reader locate the elements.
+
+The description, location and meaning of each ((ir)relevant) artwork element are provided by Google's latest Gemini Generative AI freely available through API calls. It's a handy reference for those memes one may not be familiar with at first glance. Nevertheless, it's far from perfect.
+
+### Sample
+Here is the depiction of the previous finished artwork as an example, where I highlighted with magenta numbers the elements that were not successfully identified by the AI, yet are notable:
+
+![Descriptive example](https://github.com/isaacbernat/basepaint-media/blob/main/descriptive_example.png)
+
+- 1 **Gastly**: Pokémon #0092.
+- 2 **Casper**: ghost from a 1995 movie.
+- 3 **Meme from The Sixth Sense**: (1999 movie), depicting the protagonist scared covered with a blanket.
+- 4 **Skeletor**: supervillain from 1980s franchise He-Man and the Masters of the Universe.
+- 5 **Guy Fawkes mask**: (aka Anonymous mask), a symbol of protest and rebellion.
+- 6 **Tombstone 272**: depicting the number of the artwork.
+- 7 **Anarchy symbol**: approx. since 1970s (AI thinks it's a number 4...)
+
+#### Latest AI description of 272 (gemini-2.5-flash-preview-04-17)
+- (76,39) **Skeletor**: Prominent skeletal figure in hood holding a staff, referencing the Masters of the Universe villain."
+- (27,74) **He-Man's Arm**: Muscular arm holding a staff or sword piece, referencing the Masters of the Universe hero, opponent of Skeletor."
+- (50,33) **Necromancer Figure**: Central skeletal figure wearing a wide-brimmed hat, likely the titular necromancer."
+- (23,30) **Large Skull**: A large, detailed skull facing left."
+- (40,70) **Dancing Skeleton**: A small skeleton figure with arms raised, referencing the ""Spooky Scary Skeletons"" or dancing skeleton memes."
+- (85,20) **Moth near Lamp**: A large moth-like creature hovering near a glowing circle, referencing the Moth and Lamp internet meme."
+- (75,58) **Hooded Figure**: A skeletal figure wearing a hood, resembling a Grim Reaper or other death personification."
+- (65,45) **Glowing Eye**: A single large, glowing eye, a common spooky trope, possibly referencing Sauron's eye from Lord of the Rings."
+- (45,48) **Creepy Laugh Face**: A face with a wide, unsettling smile, referencing the Creepy Laugh internet meme."
+- (14,76) **Distressed Dude**: A face with a look of discomfort or distress, often used in reaction internet memes."
+- (50,50) **Pentagram**: A five-pointed star symbol, often associated with occult practices like necromancy."
+- (10,55) **""E"" Meme**: The blocky letter ""E"", a known internet meme derived from Markiplier's Lordquaad E meme."
+- (90,10) **Ghost**: A simple sheet ghost shape, a common spooky element."
+- (85,85) **Small Skull**: A smaller, distinct skull in the lower right corner."
+- (20,50) **Small Skull**: A smaller, distinct skull on the left side below the large skull."
+- (50,5) **Background Stars**: Small star shapes in the dark background sky."
+
+#### Limitations
+There is only 1-line of text per element. This severe limitation, along with superfluous words (e.g. "small", "large", "prominent") limits the information provided. Therefore, feel free to contribute and improve them and/or the prompt/model to generate them https://github.com/isaacbernat/basepaint/issues/8
+
+Also, the descriptions in this archive may be provided by AIs and/or not be necessarily reviewed. Therefore they may contain errors, be incomplete and ultimately the responsibility one may be held accountable for is rather limited.
+
+### PDF links
+The descriptive PDF versions are available in PDF form there:
+
+- [Artwork descriptions from 001 to 100 in PDF](https://github.com/isaacbernat/basepaint-media/blob/main/pdf/basepaint_archive_0001_to_0100_descriptive.pdf)
+- [Artwork descriptions from 101 to 200 in PDF](https://github.com/isaacbernat/basepaint-media/blob/main/pdf/basepaint_archive_0101_to_0200_descriptive.pdf)
+- [Artwork descriptions from 201 to 300 in PDF](https://github.com/isaacbernat/basepaint-media/blob/main/pdf/basepaint_archive_0201_to_0300_descriptive.pdf)
+- [Artwork descriptions from 301 to 400 in PDF](https://github.com/isaacbernat/basepaint-media/blob/main/pdf/basepaint_archive_0301_to_0400_descriptive.pdf)
+- [Artwork descriptions from 401 to 500 in PDF](https://github.com/isaacbernat/basepaint-media/blob/main/pdf/basepaint_archive_0401_to_0500_descriptive.pdf)
+- [Artwork descriptions from 501 to 600 in PDF](https://github.com/isaacbernat/basepaint-media/blob/main/pdf/basepaint_archive_0501_to_0600_descriptive.pdf)
+
 ## PDF Cover
 There's a custom made cover that is suitable for both PDF editions. It contains useful statistics on most popular colours
 
@@ -46,8 +99,11 @@ There's a custom made cover that is suitable for both PDF editions. It contains
     - `metadata.csv`: csv file containing metadata for each image.
     - `videos/`: directory containing the videos in mp4 format. They condense the 24h process of creating the image.
     - `video_images/`: directory containing images in jpg format extracted from videos. Needed for the mosaic of Work In Progress pages that accompany each image in the pdf version.
-5. To create the **cover** (`-c`) and/or the **extended PDFs** with video previews (`-v`) use the appropriate parameters.
-    - E.g. `python3 create_archive.py -c -v`
+    - `description.csv`: csv file containing the description of each element for all the images.
+    - `reduced_images/`: directory containing the reduced images in png format. Used to generate the descriptions.
+
+5. To create the **cover** (`-c`) and/or the **extended PDFs** with video previews (`-v`) and/or the **descriptions** (`-d`) (with indexes `-di`) use the appropriate parameters.
+    - E.g. `python3 create_archive.py -c -v -d -di`
 
 # About
 This archive is a non-commercial, community-driven project intended for educational and historical purposes. It is **not** officially endorsed by the BasePaint team. Every effort has been made to respect the collaborative nature of BasePaint and the potential copyrights of individual creators.

config.py

@@ -2,4 +2,12 @@
 BATCH_SIZE = 100
 CREATE_COVER = False
 INCLUDE_VIDEO = False
-ARCHIVE_VERSION = "0.2.2"
+INCLUDE_DESCRIPTION = False
+EXCLUDE_IMAGES = False
+INCLUDE_DESCRIPTION_IMAGE = False
+INCLUDE_DESCRIPTION_IMAGE_GRID = False
+ARCHIVE_VERSION = "0.3.0"
+GOOGLE_API_KEY = "Replace with a valid Gemini API key in your GitHub repo secrets or locally"
+GEMINI_MODEL = "gemini-2.5-flash-preview-04-17"  #  [m for m in genai.list_models()] to check other available models
+GEMINI_SLEEP = [10, 55]  # avoid exceeding 10 RPM quota https://ai.google.dev/gemini-api/docs/rate-limits
+# Gemini usage metrics available at https://aistudio.google.com/app/usage

create_archive.py

@@ -4,18 +4,27 @@
 from fetch_metadata import create_metadata_csv
 from enrich_metadata import enrich_metadata_csv
 from image_to_pdf import create_pdf
-from config import LATEST, BATCH_SIZE, CREATE_COVER, INCLUDE_VIDEO
+from image_descriptions import create_reduced_images, create_description_csv
+from config import LATEST, BATCH_SIZE, CREATE_COVER, INCLUDE_VIDEO, INCLUDE_DESCRIPTION, EXCLUDE_IMAGES, INCLUDE_DESCRIPTION_IMAGE, INCLUDE_DESCRIPTION_IMAGE_GRID
 
 
 if __name__ == '__main__':
     parser = ArgumentParser(description='Create a Basepaint archive')
     parser.add_argument('-c', '--create-cover', action='store_true', default=CREATE_COVER, help='Create cover PDF')
     parser.add_argument('-v', '--include-video', action='store_true', default=INCLUDE_VIDEO, help='Include video frames in PDF')
+    parser.add_argument('-d', '--include-description', action='store_true', default=INCLUDE_DESCRIPTION, help='Include image descriptions')
+    parser.add_argument('-di', '--include-description-image', action='store_true', default=INCLUDE_DESCRIPTION_IMAGE, help='Include thumbnail under the image descriptions')
+    parser.add_argument('-dig', '--include-description-image-grid', action='store_true', default=INCLUDE_DESCRIPTION_IMAGE_GRID, help='Include grid on top of image descriptions')
+    parser.add_argument('-e', '--exclude-images', action='store_true', default=EXCLUDE_IMAGES, help='Exclude pages with images and metadata')
     args = parser.parse_args()
 
     print(f"Creating archive for up to day {LATEST}.")
     fetch_files(LATEST, "images")
     create_metadata_csv(LATEST)
     enrich_metadata_csv()
-    fetch_files(LATEST, "videos")
-    create_pdf(BATCH_SIZE, args.create_cover, args.include_video)
+    if args.include_video:
+        fetch_files(LATEST, "videos")
+    if args.include_description:
+        create_reduced_images()
+        create_description_csv()
+    create_pdf(BATCH_SIZE, args.create_cover, args.include_video, args.include_description, args.exclude_images, args.include_description_image, args.include_description_image_grid)

fetch_metadata.py

@@ -71,3 +71,37 @@ def create_metadata_csv(max_day: int):
                 print(f"Error processing Day {day}: {str(e)}")
     print(f"Skipped days (already in CSV): {skipped_days}")
     print("Finished creating metadata csv.")
+
+
+def load_titles(csv_path):
+    titles = {}
+    with open(csv_path, 'r', encoding='utf-8') as f:
+        reader = csv.DictReader(f)
+        for row in reader:
+            titles[int(row['NUM'])] = {
+                'title': row['TITLE'],
+                'palette': [tuple(map(int, color.strip().split(','))) for color in row['PALETTE'].split(';')],
+                'minted': row.get('MINTED', 0),
+                'artists': row.get('ARTISTS', 0),
+                'proposer': row.get('PROPOSER', ''),
+                'MINT_DATE': row.get('MINT_DATE', ''),
+            }
+    return titles
+
+
+def draw_header(canvas, day_num, titles, x_pos, page_height, page_width):
+    title_data = titles.get(day_num, {'title': '', 'palette': []})
+    title = f"Day {day_num}: {title_data['title']}"
+    canvas.setFont("MekSans-Regular", 24)
+    canvas.drawString(x_pos, page_height - 55, title)
+    
+    square_size = canvas.stringWidth("o", "MekSans-Regular", 24)
+    square_spacing = square_size * 1.2  # Add some spacing between squares
+    palette = title_data['palette']
+    total_palette_width = len(palette) * square_spacing
+    start_x = page_width - x_pos - total_palette_width
+    
+    for i, color in enumerate(palette):
+        canvas.setFillColorRGB(color[0]/255, color[1]/255, color[2]/255)
+        canvas.rect(start_x + (i * square_spacing), page_height - 50 - square_size/2, 10, 10, fill=1, stroke=1)  # stroke=1 to draw the border
+    canvas.setFillColorRGB(0, 0, 0)  # Reset fill color to black for subsequent text