Skip to content

python.md

Latest commit

 

History

History
71 lines (51 loc) · 1.83 KB

python.md

File metadata and controls

71 lines (51 loc) · 1.83 KB
Raw

Python (apify-client) — Stack Exchange Questions Scraper

Call the Stack Exchange Questions Scraper Actor from Python with the official apify-client package.

Install

pip install apify-client

Run and read results

from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("logiover/stack-exchange-questions-scraper").call(run_input={
    "site": "stackoverflow",
    "tagged": "python",
    "sort": "votes",
    "maxQuestions": 500,
})

for q in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f'[{q.get("score")} up, {q.get("viewCount")} views] {q.get("title")} - {q.get("link")}')

Newest DevOps questions for monitoring

run = client.actor("logiover/stack-exchange-questions-scraper").call(run_input={
    "site": "devops",
    "tagged": "kubernetes;helm",
    "sort": "creation",
    "maxQuestions": 300,
})

Load into pandas and rank content gaps

import pandas as pd
from apify_client import ApifyClient

client = ApifyClient("YOUR_APIFY_TOKEN")

run = client.actor("logiover/stack-exchange-questions-scraper").call(run_input={
    "site": "stackoverflow",
    "tagged": "react",
    "sort": "votes",
    "maxQuestions": 600,
})

items = list(client.dataset(run["defaultDatasetId"]).iterate_items())
df = pd.DataFrame(items)

gaps = df[(~df["hasAcceptedAnswer"]) & (df["viewCount"] > 5000)].sort_values("viewCount", ascending=False)
print(gaps[["title", "viewCount", "answerCount", "link"]].head(15))

Use an environment variable for the token

import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])

Full input/output reference is in the main README.