Skip to content

Commit 69d736b

Browse files
Seungpyo1007Seungpyo1007
committed
data(website): add 40,084 websites from Wikidata
Seeds the new `website` category (TechEngine §6.12) with the Q35127 subtree: 40,084 records, all verified:false, each citing its Wikidata item. Fields are written only where the source had them — homepage_url on 85%, plus launch_date, languages and owners where present. Nothing is inferred. Also mirrors the engine's WEBSITE_REQUIRED rule into the bundled validator {slug, name, source_urls, verified} with slug, launch_date and source-url checks. Import notes: 47,631 QIDs in the subtree yielded 40,273 English-labelled items; 189 were dropped for an unusable name, and 243 slug collisions took a -<qid> suffix. Windows-reserved stems (CON, PRN, LPT1 …) are skipped by the importer — a `con.json` from the game import previously made the tree unindexable by git. Refs #1
1 parent 5b387e4 commit 69d736b

40,085 files changed

Lines changed: 423669 additions & 0 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎app/validate.py‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,6 +119,13 @@
119119
"verified",
120120
}
121121

122+
WEBSITE_REQUIRED = {
123+
"slug",
124+
"name",
125+
"source_urls",
126+
"verified",
127+
}
128+
122129
DATE_RE = re.compile(r"^\d{4}-\d{2}-\d{2}$")
123130

124131

@@ -245,6 +252,7 @@ def validate() -> list[str]:
245252
monitors = _load("monitor")
246253
games = _load("game")
247254
software = _load("software")
255+
websites = _load("website")
248256

249257
brand_slugs = {rec["slug"] for _, rec in brands if "slug" in rec}
250258
soc_slugs = {rec["slug"] for _, rec in socs if "slug" in rec}
@@ -264,6 +272,7 @@ def validate() -> list[str]:
264272
("monitor", monitors),
265273
("game", games),
266274
("software", software),
275+
("website", websites),
267276
):
268277
_check_unique_slugs(category, records, errors)
269278

@@ -433,6 +442,13 @@ def validate() -> list[str]:
433442
if rec.get("release_date") is not None:
434443
_check_date(fname, rec["release_date"], errors)
435444

445+
for fname, rec in websites:
446+
_check_required(fname, rec, WEBSITE_REQUIRED, errors)
447+
_check_source_urls(fname, rec, errors)
448+
_check_slug(fname, rec.get("slug"), errors)
449+
if rec.get("launch_date") is not None:
450+
_check_date(fname, rec["launch_date"], errors)
451+
436452
return errors
437453

438454

‎data/website/01/010101-art-in-technological-times.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"slug": "010101-art-in-technological-times",
3+
"name": "010101: Art in Technological Times",
4+
"verified": false,
5+
"source_urls": [
6+
"https://www.wikidata.org/wiki/Q133169922"
7+
],
8+
"homepage_url": "http://archv.sfmoma.org/media/exhibitions/010101/"
9+
}

‎data/website/01/01net-com.json‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{
2+
"slug": "01net-com",
3+
"name": "01net.com",
4+
"verified": false,
5+
"source_urls": [
6+
"https://www.wikidata.org/wiki/Q2806483"
7+
],
8+
"homepage_url": "http://www.01net.com",
9+
"owners": [
10+
"Keleops"
11+
]
12+
}

‎data/website/02/02ws.json‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,16 @@
1+
{
2+
"slug": "02ws",
3+
"name": "02WS",
4+
"verified": false,
5+
"source_urls": [
6+
"https://www.wikidata.org/wiki/Q25489424"
7+
],
8+
"homepage_url": "https://www.02ws.co.il/station.php?lang=1",
9+
"launch_date": "2002-01-01",
10+
"languages": [
11+
"Hebrew",
12+
"Russian",
13+
"English",
14+
"French"
15+
]
16+
}

‎data/website/04/0404-news.json‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
{
2+
"slug": "0404-news",
3+
"name": "0404 News",
4+
"verified": false,
5+
"source_urls": [
6+
"https://www.wikidata.org/wiki/Q18191813"
7+
],
8+
"homepage_url": "https://www.0404.co.il/",
9+
"launch_date": "2012-03-01"
10+
}

‎data/website/05/05-03-1953.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"slug": "05-03-1953",
3+
"name": "05|03|1953",
4+
"verified": false,
5+
"source_urls": [
6+
"https://www.wikidata.org/wiki/Q124046509"
7+
],
8+
"homepage_url": "http://w3.osaarchivum.org/galeria/05031953/"
9+
}

‎data/website/08/08euro-ru.json‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
{
2+
"slug": "08euro-ru",
3+
"name": "08euro.Ru",
4+
"verified": false,
5+
"source_urls": [
6+
"https://www.wikidata.org/wiki/Q65807982"
7+
],
8+
"homepage_url": "http://www.08euro.ru/",
9+
"launch_date": "2008-01-01",
10+
"languages": [
11+
"Russian"
12+
]
13+
}

‎data/website/1-/1-plus-1.json‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
{
2+
"slug": "1-plus-1",
3+
"name": "1 Plus 1",
4+
"verified": false,
5+
"source_urls": [
6+
"https://www.wikidata.org/wiki/Q113187736"
7+
],
8+
"languages": [
9+
"Korean"
10+
]
11+
}

‎data/website/1/1.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"slug": "1",
3+
"name": "А1 ОН",
4+
"verified": false,
5+
"source_urls": [
6+
"https://www.wikidata.org/wiki/Q12904673"
7+
],
8+
"homepage_url": "https://a1on.mk/"
9+
}

‎data/website/10/10-daily.json‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
{
2+
"slug": "10-daily",
3+
"name": "10 daily",
4+
"verified": false,
5+
"source_urls": [
6+
"https://www.wikidata.org/wiki/Q85719581"
7+
]
8+
}

0 commit comments

Comments
 (0)