feat: unit tests

[HarishChandran3304]

Supersedes #19 - commits reworded from tests: to test: and action versions bumped.

Description

PR to add unit tests for frappe_openapi and set up the missing unit-tests CI workflow.
ref: App Spec

88 tests, all passing locally via bench --site <site> run-tests --app frappe_openapi.

Test Cases

AST helpers and type-map constants

• DEFAULT_METHODS == ["get", "post", "put", "delete"]
• _PLACEHOLDER_TYPE_MAP maps float/number → number, int/integer → integer, str/string → string, bool/boolean → boolean, list/array → array (items: {}), dict/object → object
• _PYTHON_TO_OPENAPI maps Python annotation names to OpenAPI type strings (including bytes → string)
• find_python_files returns every .py file under the base path recursively and excludes non-.py files
• get_decorator_info([]) returns (DEFAULT_METHODS, False)
• get_decorator_info parses methods=["POST"] to ["post"] (lowercased list)
• get_decorator_info parses methods="POST" (single string) to ["post"]
• get_decorator_info parses allow_guest=True and combines with methods= correctly

Docstring parsing

• _parse_typed_block returns None when no "key": <type> pairs are present
• _parse_typed_block parses placeholder pairs into a tagged {"__placeholder_schema__": ...} dict
• _parse_typed_block falls back to {"type": "string"} for unknown placeholder types
• extract_returns_from_docstring returns None for None/empty input
• extract_returns_from_docstring matches Returns: and Return: (case-insensitive)
• extract_returns_from_docstring parses a balanced {...} block as JSON (single-quotes normalised)
• extract_returns_from_docstring dispatches to _parse_typed_block for <type> placeholder dicts
• extract_returns_from_docstring returns the raw block string when JSON parsing fails
• extract_returns_from_docstring stops at the next top-level section header (e.g. Raises:) rather than consuming the rest of the docstring
• parse_docstring_args returns {} for None/empty/no-Args input
• parse_docstring_args parses name (int, optional): desc into a schema with required=False
• parse_docstring_args marks parameters without optional in the type info as required
• parse_docstring_args joins multi-line continuation descriptions with a single space
• parse_docstring_args maps list type info to {"type": "array", "items": {}}

Type annotation -> OpenAPI schema (get_openapi_type)

• None input returns {"type": "string"}
• ast.Name("int") returns {"type": "integer"}; ast.Name("list") returns array-with-items; unknown names fall back to string
• ast.Constant(None) returns {"nullable": True} (no type key)
• Optional[int] returns {"type": "integer", "nullable": True}
• List[int] returns {"type": "array", "items": {"type": "integer"}}
• Dict[str, int] returns {"type": "object"}
• Tuple[int, str] returns {"type": "array", "items": {}}
• Union[int, None] returns {"type": "integer", "nullable": True}
• Union[int, str] returns {"oneOf": [<int>, <str>]}
• Union[int, str, None] returns oneOf + nullable
• PEP 604 int | None returns {"type": "integer", "nullable": True}
• PEP 604 chained int | str | None returns oneOf + nullable
• ast.Attribute (e.g. typing.int) recurses via a synthetic ast.Name

Response schema builder (build_response_schema)

• (None, None) returns {"message": {"type": "string"}} under an object wrapper
• (None, []) returns {"message": {"type": "array", "items": {}}}
• (None, {}) returns {"message": {"type": "object"}}
• Non-empty dict example derives properties from keys (bool/int/float/str/list/dict mapped to OpenAPI types)
• int annotation with no example yields {"message": {"type": "integer"}}
• Honours a __placeholder_schema__ example and wraps it under message

Function metadata extraction (parse_functions_from_file)

• Skips functions not decorated with whitelist
• Returns the expected fields per function (name, params, doc, methods, allow_guest, return_annotation, returns_example)
• Excludes self and cls from params
• Computes required = i < n_required from len(args) - len(defaults)
• Resolves type_schema in priority order: annotation → Args: block → {"type": "string"} default

Per-app OpenAPI generation (generate_openapi_static)

• Returns {} when importlib.util.find_spec returns None
• Seed dict has openapi="3.0.0", info, paths={}, servers=[{"url": <frappe.utils.get_url()>}]
• Each whitelisted function produces an entry per HTTP method at /api/method/{app}.{module_path}.{func_name} with tags=[parent_module]
• First line of docstring becomes summary; rest becomes description (only set when non-empty)
• GET operations carry query parameters built from params (each with name, in="query", required, schema, optional description)
• POST/PUT/PATCH/DELETE operations carry an application/x-www-form-urlencoded request body with properties and required lists
• POST operation with zero parameters has no requestBody key
• POST operation whose params are all optional has requestBody.required=False
• POST operation with at least one required param has requestBody.required=True
• Non-guest operation's security is [{"TokenAuth": []}, {"bearerAuth": []}] (two separate scheme requirements, not one merged dict)
• Response content example for a placeholder dict carries placeholder defaults under message (number→0.0, integer→0, string→"", boolean→False, array→[], object→{})
• Response content example for a plain dict is {"message": <returns_example>}
• components.securitySchemes is added when at least one operation is non-guest; includes TokenAuth (apiKey, header, Authorization) and bearerAuth (http bearer, bearerFormat="JWT")
• components is omitted when every operation has allow_guest=True
• openapi["tags"] is a sorted list of {"name": <module>} dicts

App metadata and bulk generation

• get_app_title_and_version returns (hooks.app_title, module.__version__) when both are importable
• Falls back to app_name when the hooks import fails or app_title is missing
• Falls back to "1.0.0" when the app import fails or __version__ is missing
• generate_openapi_for_all_apps writes one openapi_<app>.json per enabled app under <site>/public/files/openapi/
• Overwrites info.title and info.version on the generator's output with the values from get_app_title_and_version
• Deletes any pre-existing openapi_<app>.json for disabled apps
• A delete failure is swallowed and logged via frappe.log_error with title "OpenAPI Deletion Error"
• A write failure is swallowed and logged via frappe.log_error with title "OpenAPI Generation Error"
• Progress is reported via update_progress_bar("Generating OpenAPI spec", i, total) where total = len(enabled_apps)

Custom Field lifecycle and DocType controller

• create_openapi_app_fields excludes frappe_openapi from the set of apps it processes
• Inserts a new Custom Field only for installed apps whose fieldname is NOT already present
• Inserted field has dt="OpenAPI Settings", fieldtype="Check", label=<app title>, insert_after="generate_openapi_specification_for_selected_apps_section", default="1"
• Deletes existing Custom Field rows whose fieldname is NOT in apps_to_process
• A delete failure is swallowed and logged with title "OpenAPI Custom Field Removal Error"
• A top-level exception is swallowed and logged with title "OpenAPI Custom Field Creation Error"
• after_install calls create_openapi_app_fields and swallows any exception with log title "OpenAPI After Install Error"
• before_uninstall deletes every Custom Field where dt="OpenAPI Settings", swallowing per-field and top-level exceptions with log title "OpenAPI Uninstall Error"
• OpenAPISettings.on_update enqueues generate_openapi_for_all_apps on the long queue with enqueue_after_commit=True

Website docs context (www/docs.py::get_context)

• Populates context.apps and context.app_titles only with apps whose settings flag is truthy
• When the settings doc is falsy, the gate falls open and every installed app is included
• context.default_app is the first element of enabled_apps
• context.default_app is "" when no apps are enabled

Relevant Technical Choices

• Tests in sections 1-5 use real AST parsing on small inline source strings via ast.parse (no mocks for the AST helpers, docstring parsing, type-annotation conversion, response-schema building, or function-metadata extraction).
• Tests in section 6 (generate_openapi_static) construct tempdir-based Python packages via a temp_app_package(name, files) helper in tests/__init__.py, then patch importlib.util.find_spec to point the generator at the tempdir. The generator walks real files on disk.
• Tests in section 7 (bulk generation) use a real tempdir for the output directory and write real JSON files; patches frappe.get_single, frappe.get_installed_apps, frappe.get_site_path, and the consumer-module-imported update_progress_bar to control progress and skip the real progress bar.
• Tests in sections 8 and 9 patch Frappe DB/doctype calls (frappe.get_installed_apps, frappe.get_all, frappe.get_doc, frappe.delete_doc, frappe.log_error, frappe.get_single) at the consumer module paths. A follow-up sweep (Phase 2.5) will convert these to real Custom Field fixtures.
• Per the project-wide policy, tests assert correct intended behavior, never source-as-is. The PR also bundles a separate chore: address pre-existing semgrep and ruff findings commit that fixes (or rationalises with # nosemgrep comments) the pre-existing issues that were blocking any commit on this repo, plus a real bug fix in api/examples.py where _check_demo_enabled used an untranslated frappe.throw and the demo's include_metadata flag had no effect ({} if include_metadata else {} returned {} either way).

Testing Instructions

bench --site <site> run-tests --app frappe_openapi

Additional Information:

This PR contains 4 commits on feat/unit-tests:

1. chore: address pre-existing semgrep and ruff findings — wraps frappe.throw user-facing message in frappe._(), adds # nosemgrep (with justification) to the three intentionally guest-accessible demo endpoints and the two file-traversal warnings on controlled paths, refactors filter() + comprehension into a single generator expression, replaces 15 en-dashes in comments/docstrings with hyphens, and fixes the useless {} if include_metadata else {} in demo_authenticated (now the response's metadata key is only set when include_metadata=True).
2. chore: sync tooling drift — adds the standard .github/PULL_REQUEST_TEMPLATE.md, build-test.yml, and linters.yml, and removes a stray frappe_skeleton/__init__.py line from .semgrepignore.
3. tests: add full suite for openapi spec generator and lifecycle hooks — 88 tests across 6 test files mirroring the source tree.
4. ci: add unit tests workflow — adds unit-tests.yml matching the Phase 2 pattern (ubuntu-latest, MariaDB 10.6, Redis × 2, Python 3.14, Node 24, bench run-tests --app frappe_openapi).

Screenshot/Screencast

Checklist

• I have carefully reviewed the code before submitting it for review.
• This code is adequately covered by unit tests to validate its functionality.
• I have conducted thorough testing to ensure it functions as intended.
• A member of the QA team has reviewed and tested this PR (To be checked by QA or code reviewer)

Fixes #

[Copilot]

Pull request overview

Adds a comprehensive unit/integration-style test suite for frappe_openapi’s OpenAPI generator and lifecycle hooks, and introduces CI workflows to run those tests (plus lint/build checks) consistently in GitHub Actions. This strengthens regression protection around spec generation, app enablement logic, and install/uninstall behavior.

Changes:

• Added ~88 tests covering AST/docstring parsing, schema generation, app metadata/bulk generation, and settings/docs context behavior.
• Added GitHub Actions workflows for unit tests, linters, and bench build verification, plus a standard PR template.
• Included small hardening/cleanup changes (semgrep suppressions for non-user-controlled file paths; demo API error message translation + metadata response fix).

Reviewed changes

Copilot reviewed 14 out of 19 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
frappe_openapi/tests/www/test_docs.py	Tests www/docs.py::get_context app filtering/default selection behavior.
frappe_openapi/tests/www/__init__.py	Marks tests.www as a package for test discovery/imports.
frappe_openapi/tests/test_generate_api_docs.py	Core test suite for generator helpers and OpenAPI generation/bulk write logic.
frappe_openapi/tests/setup/test_uninstall.py	Tests uninstall hook behavior (Custom Field cleanup + error logging).
frappe_openapi/tests/setup/test_install.py	Tests install hook behavior (calls + error logging).
frappe_openapi/tests/setup/__init__.py	Marks tests.setup as a package.
frappe_openapi/tests/doctype/openapi_settings/test_openapi_settings.py	Tests OpenAPISettings.on_update enqueues regeneration correctly.
frappe_openapi/tests/doctype/openapi_settings/__init__.py	Marks doctype test package.
frappe_openapi/tests/doctype/__init__.py	Marks tests.doctype as a package.
frappe_openapi/tests/config/test_create_app_list.py	Tests Custom Field lifecycle for per-app OpenAPI enable flags.
frappe_openapi/tests/config/__init__.py	Marks tests.config as a package.
frappe_openapi/tests/__init__.py	Adds shared AST/temp-package helpers used across tests.
frappe_openapi/frappe_openapi/generate_api_docs.py	Minor internal adjustments (string join refactor; semgrep suppressions on controlled paths).
frappe_openapi/api/examples.py	Improves demo endpoint behavior (translated throw message; correct metadata inclusion) and adds semgrep suppressions for intended guest endpoints.
.semgrepignore	Removes an unrelated ignored path entry.
.github/workflows/unit-tests.yml	Adds unit-test CI workflow (bench setup + run-tests).
.github/workflows/linters.yml	Adds linting workflow using pre-commit on changed files.
.github/workflows/build-test.yml	Adds bench build verification workflow.
.github/PULL_REQUEST_TEMPLATE.md	Adds a standard PR template.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.