feat(async): introduce async module#1531
Open
OkoyaUsman wants to merge 11 commits intobrowniebroke:mainfrom
Open
feat(async): introduce async module#1531OkoyaUsman wants to merge 11 commits intobrowniebroke:mainfrom
OkoyaUsman wants to merge 11 commits intobrowniebroke:mainfrom
Conversation
Currently using this on a project and Async is super important to me, took some time to write the async module.
Contributor
Reviewer's GuideIntroduces a new async client package Sequence diagram for async pagination with AsyncPaginatedListsequenceDiagram
actor Developer
participant AsyncClient
participant AsyncPaginatedList
participant httpx_AsyncClient as httpx_AsyncClient
participant DeezerAPI
Developer->>AsyncClient: get_user_tracks(user_id)
AsyncClient->>AsyncClient: _get_paginated_list("user/{id}/tracks")
AsyncClient-->>Developer: AsyncPaginatedList
Developer->>AsyncPaginatedList: async for track in tracks
loop iterate_loaded_elements
AsyncPaginatedList-->>Developer: yield cached track
end
alt more_pages_available
AsyncPaginatedList->>AsyncPaginatedList: _could_grow()
AsyncPaginatedList->>AsyncPaginatedList: _grow()
AsyncPaginatedList->>AsyncPaginatedList: _fetch_next_page()
AsyncPaginatedList->>AsyncClient: request("GET", next_path, params)
AsyncClient->>httpx_AsyncClient: request("GET", next_path, params)
httpx_AsyncClient->>DeezerAPI: HTTP GET next_page_url
DeezerAPI-->>httpx_AsyncClient: JSON page payload
httpx_AsyncClient-->>AsyncClient: httpx.Response
AsyncClient->>AsyncClient: response.raise_for_status()
AsyncClient->>AsyncClient: _process_json(json_data, paginate_list=True)
AsyncClient-->>AsyncPaginatedList: list of ResourceType
AsyncPaginatedList->>AsyncPaginatedList: extend _elements
AsyncPaginatedList-->>Developer: yield new track
end
Class diagram for async_deezer core typesclassDiagram
class AsyncClient {
+dict~str, type Resource or None~ objects_types
+AsyncClient(access_token: str, headers: HeaderTypes)
+request(method: str, path: str, parent: Resource, resource_type: type Resource, resource_id: int, paginate_list: bool, kwargs: Any)
+get_album(album_id: int) Album
+get_artist(artist_id: int) Artist
+get_chart(genre_id: int) Chart
+get_tracks_chart(genre_id: int) list~Track~
+get_albums_chart(genre_id: int) list~Album~
+get_artists_chart(genre_id: int) list~Artist~
+get_playlists_chart(genre_id: int) list~Playlist~
+get_podcasts_chart(genre_id: int) list~Podcast~
+get_editorial(editorial_id: int) Editorial
+list_editorials() AsyncPaginatedList~Editorial~
+get_episode(episode_id: int) Episode
+get_genre(genre_id: int) Genre
+list_genres() list~Genre~
+get_playlist(playlist_id: int) Playlist
+get_podcast(podcast_id: int) Podcast
+get_radio(radio_id: int) Radio
+list_radios() list~Radio~
+get_radios_top() AsyncPaginatedList~Radio~
+get_track(track_id: int) Track
+get_user(user_id: int) User
+get_user_recommended_tracks(kwargs: Any) AsyncPaginatedList~Track~
+get_user_recommended_albums(kwargs: Any) AsyncPaginatedList~Album~
+get_user_recommended_artists(kwargs: Any) AsyncPaginatedList~Artist~
+get_user_recommended_playlists(kwargs: Any) AsyncPaginatedList~Playlist~
+get_user_flow(kwargs: Any) AsyncPaginatedList~Track~
+get_user_albums(user_id: int) AsyncPaginatedList~Album~
+add_user_album(album_id: int) bool
+remove_user_album(album_id: int) bool
+get_user_artists(user_id: int) AsyncPaginatedList~Artist~
+add_user_artist(artist_id: int) bool
+remove_user_artist(artist_id: int) bool
+get_user_followers(user_id: int) AsyncPaginatedList~User~
+get_user_followings(user_id: int) AsyncPaginatedList~User~
+add_user_following(user_id: int) bool
+remove_user_following(user_id: int) bool
+get_user_history() AsyncPaginatedList~Track~
+get_user_tracks(user_id: int) AsyncPaginatedList~Track~
+add_user_track(track_id: int) bool
+remove_user_track(track_id: int) bool
+remove_user_playlist(playlist_id: int) bool
+add_user_playlist(playlist_id: int) bool
+create_playlist(playlist_name: str) int
+delete_playlist(playlist_id: int) bool
+search(query: str, strict: bool, ordering: str, artist: str, album: str, track: str, label: str, dur_min: int, dur_max: int, bpm_min: int, bpm_max: int) AsyncPaginatedList~Track~
+search_albums(query: str, strict: bool, ordering: str) AsyncPaginatedList~Album~
+search_artists(query: str, strict: bool, ordering: str) AsyncPaginatedList~Artist~
+search_playlists(query: str, strict: bool, ordering: str) AsyncPaginatedList~Playlist~
}
class httpx_AsyncClient {
}
AsyncClient --|> httpx_AsyncClient
class AsyncPaginatedList~ResourceType~ {
-list~ResourceType~ _elements
-AsyncClient _client
-str _base_path
-dict _base_params
-str _next_path
-dict _next_params
-Resource _parent
-int _total
+AsyncPaginatedList(client: AsyncClient, base_path: str, parent: Resource, params: dict)
+__aiter__() AsyncGenerator~ResourceType, None~
+aget(index: int) ResourceType
+aslice(start: int, stop: int) list~ResourceType~
+get_total() int
}
class Resource {
+int id
+str type
-tuple~str~ _fields
+Resource(client: AsyncClient, json: dict~str, Any~)
+as_dict() dict~str, Any~
+get_relation(relation: str, resource_type: type Resource, params: dict, fwd_parent: bool)
+post_relation(relation: str, params: dict)
+delete_relation(relation: str, params: dict)
+get_paginated_list(relation: str, params: dict) AsyncPaginatedList
+get()
}
AsyncClient o--> Resource : creates
AsyncClient ..> AsyncPaginatedList : returns
Resource ..> AsyncPaginatedList : returns
AsyncPaginatedList --> AsyncClient : uses
class Album {
+int id
+str title
+str upc
+str link
+str share
+str cover
+str cover_small
+str cover_medium
+str cover_big
+str cover_xl
+str md5_image
+int genre_id
+list~Genre~ genres
+str label
+int nb_tracks
+int duration
+int fans
+dt_date release_date
+str record_type
+bool available
+Album alternative
+str tracklist
+bool explicit_lyrics
+int explicit_content_lyrics
+int explicit_content_cover
+list~Artist~ contributors
+Artist artist
+list~Track~ tracks
+get_artist() Artist
+get_tracks(kwargs: Any) AsyncPaginatedList~Track~
}
class Artist {
+int id
+str name
+str link
+str share
+str picture
+str picture_small
+str picture_medium
+str picture_big
+str picture_xl
+int nb_album
+int nb_fan
+bool radio
+str tracklist
+get_top(kwargs: Any) AsyncPaginatedList~Track~
+get_related(kwargs: Any) AsyncPaginatedList~Artist~
+get_radio(kwargs: Any) list~Track~
+get_albums(kwargs: Any) AsyncPaginatedList~Album~
+get_playlists(kwargs: Any) AsyncPaginatedList~Playlist~
}
class Track {
+int id
+bool readable
+str title
+str title_short
+str title_version
+bool unseen
+str isrc
+str link
+str share
+int duration
+int track_position
+int disk_number
+int rank
+dt_date release_date
+bool explicit_lyrics
+int explicit_content_lyrics
+int explicit_content_cover
+str preview
+float bpm
+float gain
+list~str~ available_countries
+Track alternative
+list~Artist~ contributors
+str md5_image
+Artist artist
+Album album
+get_artist() Artist
+get_album() Album
}
class User {
+int id
+str name
+str lastname
+str firstname
+str email
+int status
+dt_date birthday
+dt_date inscription_date
+str gender
+str link
+str picture
+str picture_small
+str picture_medium
+str picture_big
+str picture_xl
+str country
+str lang
+bool is_kid
+str explicit_content_level
+list~str~ explicit_content_levels_available
+str tracklist
+get_albums(params: Any) AsyncPaginatedList~Album~
+add_album(album: Album)
+remove_album(album: Album)
+get_tracks(kwargs: Any) AsyncPaginatedList~Track~
+add_track(track: Track)
+remove_track(track: Track)
+get_artists(params: Any) AsyncPaginatedList~Artist~
+add_artist(artist: Artist)
+remove_artist(artist: Artist)
+get_followers(params: Any) AsyncPaginatedList~User~
+get_followings(params: Any) AsyncPaginatedList~User~
+follow(user: User)
+unfollow(user: User)
+get_playlists(params: Any) AsyncPaginatedList~Playlist~
+add_playlist(playlist: Playlist)
+remove_playlist(playlist: Playlist)
+create_playlist(title: str) int
}
class Playlist {
+int id
+str title
+str description
+int duration
+bool public
+bool is_loved_track
+bool collaborative
+int nb_tracks
+int unseen_track_count
+int fans
+str link
+str share
+str picture
+str picture_small
+str picture_medium
+str picture_big
+str picture_xl
+str checksum
+User creator
+list~Track~ tracks
+get_tracks(kwargs: Any) AsyncPaginatedList~Track~
+get_fans(kwargs: Any) AsyncPaginatedList~User~
+mark_seen() bool
+add_tracks(tracks: Iterable)
+delete_tracks(tracks: Iterable)
+reorder_tracks(order: Iterable)
}
class Episode {
+int id
+str title
+str description
+bool available
+dt_datetime release_date
+int duration
+str link
+str share
+str picture
+str picture_small
+str picture_medium
+str picture_big
+str picture_xl
+Podcast podcast
+add_bookmark(offset: int) bool
+remove_bookmark() bool
}
class Genre {
+int id
+str name
+str picture
+str picture_small
+str picture_medium
+str picture_big
+str picture_xl
+get_artists(kwargs: Any) list~Artist~
+get_podcasts(kwargs: Any) AsyncPaginatedList~Podcast~
+get_radios(kwargs: Any) list~Radio~
}
class Editorial {
+int id
+str name
+str picture
+str picture_small
+str picture_medium
+str picture_big
+str picture_xl
+get_selection() list~Album~
+get_chart() Chart
+get_releases(kwargs: Any) AsyncPaginatedList~Album~
}
class Podcast {
+int id
+str title
+str description
+bool available
+int fans
+str link
+str share
+str picture
+str picture_small
+str picture_medium
+str picture_big
+str picture_xl
+get_episodes(kwargs: Any) AsyncPaginatedList~Episode~
}
class Radio {
+int id
+str title
+str description
+str share
+str picture
+str picture_small
+str picture_medium
+str picture_big
+str picture_xl
+str tracklist
+str md5_image
+get_tracks() list~Track~
}
Album --|> Resource
Artist --|> Resource
Track --|> Resource
User --|> Resource
Playlist --|> Resource
Episode --|> Resource
Genre --|> Resource
Editorial --|> Resource
Podcast --|> Resource
Radio --|> Resource
Album --> Artist
Album --> Track
Album --> Genre
Track --> Artist
Track --> Album
Playlist --> Track
Playlist --> User
Episode --> Podcast
Editorial --> Chart
Genre --> Podcast
Genre --> Radio
Radio --> Track
User --> Playlist
User --> Album
User --> Track
User --> Artist
Flow diagram for AsyncPaginatedList data loading strategyflowchart TD
A_start[Start async iteration or helper call
aget, aslice, get_total] --> B_check_cached{Enough elements
in _elements?}
B_check_cached -- yes --> C_return_cached[Return data from _elements]
B_check_cached -- no --> D_can_grow{_next_path is not None?}
D_can_grow -- no --> E_end[Stop iteration or return
available elements]
D_can_grow -- yes --> F_fetch_page[Call _fetch_next_page]
F_fetch_page --> G_request[AsyncClient.request
GET next_path]
G_request --> H_httpx[httpx.AsyncClient.request]
H_httpx --> I_api[Deezer API
returns JSON page]
I_api --> J_process[AsyncClient._process_json
with paginate_list = True]
J_process --> K_payload[response_payload with
data, total, next]
K_payload --> L_update_state[Update _elements,
_total, _next_path,
_next_params]
L_update_state --> B_check_cached
C_return_cached --> M_done[Result to caller]
E_end --> M_done
File-Level Changes
Possibly linked issues
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #1531 +/- ##
=======================================
Coverage 99.86% 99.86%
=======================================
Files 20 20
Lines 723 723
Branches 46 46
=======================================
Hits 722 722
Partials 1 1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
CodeQL found more than 20 potential problems in the proposed changes. Check the Files changed tab for more details.
![sourcery-ai[bot]](https://avatars.githubusercontent.com/in/48477?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Hey - I've found 6 issues, and left some high level feedback:
- There is a circular import between
async_deezer.client.AsyncClientandasync_deezer.pagination.AsyncPaginatedList(each imports the other at module import time); consider moving theAsyncClientimport inpagination.pybehindTYPE_CHECKINGor using string-based type hints to break the cycle. - In
async_deezer.resources.Episode,_infer_missing_fieldcallssuper()._infer_missing_field, but the asyncResourcebase class does not define this method, which will raise at runtime; either add_infer_missing_fieldto the asyncResourceor drop thesuper()call and handle all cases locally. - In
async_deezer.pagination.AsyncPaginatedList,parentis typed asdeezer.resources.Resourcewhile the async resources inherit fromasync_deezer.resources.Resource, which can be confusing and makes static typing inaccurate; aligning the type to the asyncResource(and adjusting imports accordingly) would make the async layer more consistent.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- There is a circular import between `async_deezer.client.AsyncClient` and `async_deezer.pagination.AsyncPaginatedList` (each imports the other at module import time); consider moving the `AsyncClient` import in `pagination.py` behind `TYPE_CHECKING` or using string-based type hints to break the cycle.
- In `async_deezer.resources.Episode`, `_infer_missing_field` calls `super()._infer_missing_field`, but the async `Resource` base class does not define this method, which will raise at runtime; either add `_infer_missing_field` to the async `Resource` or drop the `super()` call and handle all cases locally.
- In `async_deezer.pagination.AsyncPaginatedList`, `parent` is typed as `deezer.resources.Resource` while the async resources inherit from `async_deezer.resources.Resource`, which can be confusing and makes static typing inaccurate; aligning the type to the async `Resource` (and adjusting imports accordingly) would make the async layer more consistent.
## Individual Comments
### Comment 1
<location> `src/async_deezer/resources/episode.py:36-41` </location>
<code_context>
+
+ _parse_release_date = staticmethod(parse_datetime)
+
+ def _infer_missing_field(self, item) -> Any: # type: ignore[override]
+ if item == "link":
+ return f"https://www.deezer.com/episode/{self.id}"
+ elif item == "share":
+ return f"{self.link}?utm_source=deezer&utm_content=episode-{self.id}&utm_medium=web"
+ return super()._infer_missing_field(item) # type: ignore[no-any-return]
+
+ async def add_bookmark(self, offset: int) -> bool:
</code_context>
<issue_to_address>
**issue (bug_risk):** This override calls a base method that does not exist on Resource, which will raise at runtime if invoked.
`Episode` implements `_infer_missing_field`, but `Resource` doesn’t define it, so `super()._infer_missing_field(item)` will raise `AttributeError` if reached. Please either add `_infer_missing_field` to `Resource` (consistent with the sync variant) or remove this override and set `link`/`share` defaults elsewhere (e.g. in `__init__`).
</issue_to_address>
### Comment 2
<location> `src/async_deezer/resources/resource.py:9-18` </location>
<code_context>
+class Resource:
</code_context>
<issue_to_address>
**suggestion (bug_risk):** Resource is missing the `_infer_missing_field` hook that some resource types expect to use for inferred attributes.
Async resources like `Episode` rely on `_infer_missing_field` from the sync base class to populate defaults (e.g. `link`, `share`). In this async `Resource`, that hook doesn’t exist, so subclasses calling `super()._infer_missing_field` will raise `AttributeError` and inferred fields won’t be set. To keep behavior consistent with the sync API, either port the `_infer_missing_field` / `__getattr__` logic here or update subclasses to no longer depend on it.
Suggested implementation:
```python
from typing import Any
from async_deezer.pagination import AsyncPaginatedList
from deezer.resources import Resource as SyncResource
```
```python
class Resource:
"""
Async base class for any Deezer resource.
This mirrors :class:`deezer.resources.Resource`, but relation helpers and
``get`` are implemented as async methods, and pagination returns
:class:`async_deezer.pagination.AsyncPaginatedList`.
"""
id: int
type: str
def _infer_missing_field(self, name: str) -> Any:
"""
Hook used by __getattr__ and subclasses to lazily infer missing fields.
This delegates to the synchronous Resource implementation to keep
behavior (e.g. inferred ``link`` / ``share``) consistent with the
sync API.
"""
return SyncResource._infer_missing_field(self, name)
def __getattr__(self, name: str) -> Any:
"""
Fallback attribute access that mirrors the synchronous Resource.
When an attribute is missing, attempt to infer it via
:meth:`_infer_missing_field`. If inference fails, raise AttributeError.
"""
return SyncResource.__getattr__(self, name)
```
If other parts of this file override `__getattr__` or `_infer_missing_field` later on, those overrides should either be removed in favor of this delegation, or updated to call `super().__getattr__(name)` / `super()._infer_missing_field(name)` so that the synchronous inference behavior is preserved.
</issue_to_address>
### Comment 3
<location> `src/async_deezer/client.py:51-53` </location>
<code_context>
+ "playlist": Playlist,
+ "podcast": Podcast,
+ "radio": Radio,
+ "search": None,
+ "track": Track,
+ "user": User,
+ }
+
</code_context>
<issue_to_address>
**issue (bug_risk):** The objects_types entry for 'search' is None but later code asserts that the resolved class is not None, which can cause an assertion failure.
Because `objects_types` defines `
</issue_to_address>
### Comment 4
<location> `src/async_deezer/client.py:385-389` </location>
<code_context>
+ query_parts.append(query)
+ query_parts.extend(
+ f'{param_name}:"{param_value}"'
+ for param_name, param_value in advanced_params.items()
+ if param_value
+ )
+
</code_context>
<issue_to_address>
**suggestion:** Filtering advanced search parameters by simple truthiness may unintentionally drop valid values like 0.
In `_search`, advanced params are only added when `param_value` is truthy, so values like `0` for `dur_min/dur_max/bpm_min/bpm_max` are dropped from the query. To allow falsy-but-valid values, change the condition to `if param_value is not None` so only `None` is excluded.
```suggestion
query_parts.extend(
f'{param_name}:"{param_value}"'
for param_name, param_value in advanced_params.items()
if param_value is not None
)
```
</issue_to_address>
### Comment 5
<location> `README.md:58-59` </location>
<code_context>
-## Basic Use
+## Basic Use (synchronous)
Easily query the Deezer API from you Python code. The data returned by the Deezer
API is mapped to python resources:
</code_context>
<issue_to_address>
**issue (typo):** Typo in phrase "from you Python code"; should be "from your Python code".
Update the sentence to: "Easily query the Deezer API from your Python code."
```suggestion
Easily query the Deezer API from your Python code. The data returned by the Deezer
API is mapped to python resources:
```
</issue_to_address>
### Comment 6
<location> `README.md:107` </location>
<code_context>
+ project so it does not change the public API of `deezer-python`.
+
Ready for more? Look at our whole [documentation](http://deezer-python.readthedocs.io/)
on Read The Docs or have a play in pre-populated Jupyter notebook
[on Binder](https://mybinder.org/v2/gh/browniebroke/deezer-python/main?filepath=demo.ipynb).
</code_context>
<issue_to_address>
**suggestion (typo):** Missing article before "pre-populated Jupyter notebook".
Suggest: "have a play in a pre-populated Jupyter notebook" (adding the article "a").
</issue_to_address>Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.
OkoyaUsman
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Currently using this on a project and Async is super important to me, took some time to write the async module.
Description of change
Implementation of #1339
This PR adds an async version of the Deezer Python client, available alongside the existing synchronous client.
What's New
Features
Pull-Request Checklist
mainbranchImproves #1339Summary by Sourcery
Add an asynchronous Deezer client alongside the existing synchronous client, exposing async resource APIs and pagination while documenting how to use them.
New Features:
Documentation: