Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/lint.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ jobs:
strategy:
matrix:
python-version:
["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t"]
["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t", "3.15.0-alpha.1"]

steps:
- uses: actions/checkout@v4
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/tests_and_coverage.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ jobs:
matrix:
os: [macos-latest, ubuntu-latest, windows-latest]
python-version:
["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t"]
["3.8", "3.9", "3.10", "3.11", "3.12", "3.13", "3.14", "3.14t", "3.15.0-alpha.1"]

steps:
- uses: actions/checkout@v4
Expand Down
34 changes: 17 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,22 @@

[![Downloads](https://static.pepy.tech/badge/dirstree/month)](https://pepy.tech/project/dirstree)
[![Downloads](https://static.pepy.tech/badge/dirstree)](https://pepy.tech/project/dirstree)
[![Coverage Status](https://coveralls.io/repos/github/pomponchik/dirstree/badge.svg?branch=main)](https://coveralls.io/github/pomponchik/dirstree?branch=main)
[![Lines of code](https://sloc.xyz/github/pomponchik/dirstree/?category=code)](https://github.com/boyter/scc/)
[![Hits-of-Code](https://hitsofcode.com/github/pomponchik/dirstree?branch=main)](https://hitsofcode.com/github/pomponchik/dirstree/view?branch=main)
[![Test-Package](https://github.com/pomponchik/dirstree/actions/workflows/tests_and_coverage.yml/badge.svg)](https://github.com/pomponchik/dirstree/actions/workflows/tests_and_coverage.yml)
[![Coverage Status](https://coveralls.io/repos/github/mutating/dirstree/badge.svg?branch=main)](https://coveralls.io/github/mutating/dirstree?branch=main)
[![Lines of code](https://sloc.xyz/github/mutating/dirstree/?category=code)](https://github.com/boyter/scc/)
[![Hits-of-Code](https://hitsofcode.com/github/mutating/dirstree?branch=main)](https://hitsofcode.com/github/mutating/dirstree/view?branch=main)
[![Test-Package](https://github.com/mutating/dirstree/actions/workflows/tests_and_coverage.yml/badge.svg)](https://github.com/mutating/dirstree/actions/workflows/tests_and_coverage.yml)
[![Python versions](https://img.shields.io/pypi/pyversions/dirstree.svg)](https://pypi.python.org/pypi/dirstree)
[![PyPI version](https://badge.fury.io/py/dirstree.svg)](https://badge.fury.io/py/dirstree)
[![Checked with mypy](http://www.mypy-lang.org/static/mypy_badge.svg)](http://mypy-lang.org/)
[![Ruff](https://img.shields.io/endpoint?url=https://raw.githubusercontent.com/astral-sh/ruff/main/assets/badge/v2.json)](https://github.com/astral-sh/ruff)
[![DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/pomponchik/dirstree)
[![DeepWiki](https://deepwiki.com/badge.svg)](https://deepwiki.com/mutating/dirstree)

</details>

![logo](https://raw.githubusercontent.com/pomponchik/dirstree/develop/docs/assets/logo_1.svg)
![logo](https://raw.githubusercontent.com/mutating/dirstree/develop/docs/assets/logo_1.svg)


There are many libraries for traversing directories. You can also do this using the standard library. This particular library is a bit different in that:
There are many libraries for traversing directories. You can also do this using the standard library. What makes this library different:

- ⚗️ Filtering by file extensions, text patterns in [`.gitignore` format](https://git-scm.com/book/en/v2/Git-Basics-Recording-Changes-to-the-Repository#_ignoring), and using custom callables.
- 🐍 Natively works with both [`Path` objects](https://docs.python.org/3/library/pathlib.html#basic-use) from the standard library and strings.
Expand All @@ -37,23 +37,23 @@ There are many libraries for traversing directories. You can also do this using

## Installation

You can install [`dirstree`](https://pypi.python.org/pypi/dirstree) using pip:
You can install [`dirstree`](https://pypi.org/project/dirstree/) with `pip`:

```bash
pip install dirstree
```

You can also quickly try out this and other packages without having to install using [instld](https://github.com/pomponchik/instld).
You can also use [`instld`](https://github.com/pomponchik/instld) to quickly try out this package and others without installing them.


## Basic usage

It's very easy to work with the library in your own code:
The library is easy to use:

- Create a crawler object, passing the path to the base directory and, if necessary, additional arguments.
- Iterate through it.

The simplest code example would look like this:
The simplest example would look like this:

```python
from dirstree import Crawler
Expand All @@ -64,12 +64,12 @@ for file in crawler:
print(file)
```

> ↑ Here we output recursively (that is, including the contents of nested directories) all files from the current directory. At each iteration, we get a new [`Path` object](https://docs.python.org/3/library/pathlib.html#basic-use).
> ↑ This recursively prints all files in the current directory, including files in nested directories. At each iteration, we get a new [`Path` object](https://docs.python.org/3/library/pathlib.html#basic-use).


## Filtering

Iterating through the files in the directory, you may not want to view all files, but only files of a certain type. To do this, ignore all other files. How to do it? There are 3 ways:
Iterating through the files in the directory, you may not want to view all files, but only files of a certain type. To do this, ignore all other files. How to do it? There are three ways:

- Bypass only files with the specified [extensions](https://en.wikipedia.org/wiki/Filename_extension), such as `.txt`, `.doc`, or `.py`.
- Bypass files whose paths follow a specific text pattern.
Expand Down Expand Up @@ -103,15 +103,15 @@ crawler = Crawler('.', exclude=['.git', 'venv']) # Exclude ".git" and "venv" di
If you need a universal way to filter out unnecessary paths, pass your function as the `filter` parameter:

```python
crawler = Crawler('.', filter = lambda path: len(str(path)) == 7) # Iterate only on paths that are 7 characters long.
crawler = Crawler('.', filter=lambda path: len(str(path)) == 7) # Iterate only on paths that are 7 characters long.
```


## Working with Cancellation Tokens

You can set an arbitrary condition under which file traversal will stop using [cancellation tokens](https://cantok.readthedocs.io/en/latest/the_pattern/) from the [`cantok`](https://github.com/pomponchik/cantok) library.

> There are 2 ways to do this ↓
> There are two ways to do this ↓

1. If you use the crawler as a one-time object for a single iteration, set the token when creating it:

Expand All @@ -120,7 +120,7 @@ for path in Crawler('.', token=TimeoutToken(0.0001)): # Limit the iteration time
print(path)
```

2. If you plan to use the crawler object several times, use the `go()` method for iteration and pass a new token to it everytime:
2. If you plan to use the crawler object several times, use the `go()` method for iteration and pass a new token to it every time:

```python
crawler = Crawler('.')
Expand All @@ -141,7 +141,7 @@ for path in Crawler('../dirstree') + Crawler('../cantok'):
print(path)
```

> ↑ The paths that you will iterate on will be automatically deduplicated.
> ↑ The paths that you will iterate over will be automatically deduplicated.

> ↑ You can also impose arbitrary restrictions on each of the summed objects, all of them will be taken into account.

Expand Down
4 changes: 2 additions & 2 deletions dirstree/crawlers/crawler.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

import pathspec
from cantok import AbstractToken, DefaultToken
from printo import descript_data_object, not_none
from printo import describe_data_object, not_none
from sigmatch import PossibleCallMatcher

from dirstree.crawlers.abstract import AbstractCrawler
Expand Down Expand Up @@ -60,7 +60,7 @@ def __repr__(self) -> str:
}
filters.update(self.addictional_repr_filters)

return descript_data_object(
return describe_data_object(
self.__class__.__name__,
self.paths,
{
Expand Down
4 changes: 2 additions & 2 deletions dirstree/crawlers/group.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
from typing import Generator, List

from cantok import AbstractToken, DefaultToken
from printo import descript_data_object
from printo import describe_data_object

from dirstree.crawlers.abstract import AbstractCrawler

Expand All @@ -12,7 +12,7 @@ def __init__(self, crawlers: List[AbstractCrawler]) -> None:
self.crawlers = crawlers

def __repr__(self) -> str:
return descript_data_object(
return describe_data_object(
type(self).__name__,
(self.crawlers,),
{},
Expand Down
15 changes: 10 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,13 +4,17 @@ build-backend = "setuptools.build_meta"

[project]
name = "dirstree"
version = "0.0.4"
version = "0.0.5"
authors = [{ name = "Evgeniy Blinov", email = "zheni-b@yandex.ru" }]
description = 'Another library for iterating through the contents of a directory'
readme = "README.md"
requires-python = ">=3.8"
dependencies = ['pathspec==0.12.1', 'printo>=0.0.10', 'cantok==0.0.32', 'sigmatch>=0.0.6']

dependencies = [
'printo>=0.0.22',
'cantok==0.0.32',
'sigmatch>=0.0.8',
'pathspec==0.12.1',
]
classifiers = [
"Operating System :: OS Independent",
'Operating System :: MacOS :: MacOS X',
Expand All @@ -25,6 +29,7 @@ classifiers = [
'Programming Language :: Python :: 3.12',
'Programming Language :: Python :: 3.13',
'Programming Language :: Python :: 3.14',
'Programming Language :: Python :: 3.15',
'Programming Language :: Python :: Free Threading',
'Programming Language :: Python :: Free Threading :: 3 - Stable',
'License :: OSI Approved :: MIT License',
Expand All @@ -49,5 +54,5 @@ lint.select = ["ERA001", "YTT", "ASYNC", "BLE", "B", "A", "COM", "INP", "PIE", "
format.quote-style = "single"

[project.urls]
'Source' = 'https://github.com/pomponchik/dirstree'
'Tracker' = 'https://github.com/pomponchik/dirstree/issues'
'Source' = 'https://github.com/mutating/dirstree'
'Tracker' = 'https://github.com/mutating/dirstree/issues'
2 changes: 1 addition & 1 deletion tests/test_crawler.py
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ def test_crawl_test_directory_with_exclude_patterns_and_extensions(
(Crawler('.', exclude=['*.py'], extensions=['.py']), "Crawler('.', extensions=['.py'], exclude=['*.py'])"),
(Crawler('.', exclude=['*.py']), "Crawler('.', exclude=['*.py'])"),
(Crawler('.', filter=custom_filter), "Crawler('.', filter=custom_filter)"),
(Crawler('.', filter=lambda x: True), "Crawler('.', filter=λ)"), # noqa: ARG005
(Crawler('.', filter=lambda x: True), "Crawler('.', filter=lambda x: True)"), # noqa: ARG005
(Crawler('.', token=ConditionToken(lambda: True)), "Crawler('.', token=ConditionToken(λ))"),
(Crawler('../dirstree') + Crawler('../cantok'), "CrawlersGroup([Crawler('../dirstree'), Crawler('../cantok')])"),
(Crawler('../dirstree') + PythonCrawler('../cantok'), "CrawlersGroup([Crawler('../dirstree'), PythonCrawler('../cantok')])"),
Expand Down
2 changes: 1 addition & 1 deletion tests/test_python_crawler.py
Original file line number Diff line number Diff line change
Expand Up @@ -74,7 +74,7 @@ def test_crawl_test_directory_with_exclude_inits(
(PythonCrawler('usr/bin'), "PythonCrawler('usr/bin')"),
(PythonCrawler('.', exclude=['*.py']), "PythonCrawler('.', exclude=['*.py'])"),
(PythonCrawler('.', filter=custom_filter), "PythonCrawler('.', filter=custom_filter)"),
(PythonCrawler('.', filter=lambda x: True), "PythonCrawler('.', filter=λ)"), # noqa: ARG005
(PythonCrawler('.', filter=lambda x: True), "PythonCrawler('.', filter=lambda x: True)"), # noqa: ARG005
(PythonCrawler('.', token=ConditionToken(lambda: True)), "PythonCrawler('.', token=ConditionToken(λ))"),
(PythonCrawler('../dirstree') + PythonCrawler('../cantok'), "CrawlersGroup([PythonCrawler('../dirstree'), PythonCrawler('../cantok')])"),
],
Expand Down
Loading