Complete guide for installing, developing, and publishing PipeFrame.
Author: Dr. Yasser Mustafa
Email: yasser.mustafan@gmail.com
Repository: https://github.com/Yasser03/pipeframe
# Install from PyPI (once published)
pip install pipeframe
# With all optional dependencies
pip install pipeframe[all]
# With specific features
pip install pipeframe[excel] # Excel support
pip install pipeframe[parquet] # Parquet files
pip install pipeframe[sql] # SQL databases
pip install pipeframe[plot] # Visualization# Clone the repository
git clone https://github.com/Yasser03/pipeframe.git
cd pipeframe
# Install in development mode
pip install -e ".[dev,test]"import pipeframe
print(pipeframe.__version__) # Should print: 0.2.0
# Run quickstart
python examples/quickstart.py# Fork on GitHub, then:
git clone https://github.com/YOUR_USERNAME/pipeframe.git
cd pipeframe# Using venv
python3 -m venv venv
source venv/bin/activate # On Windows: venv\Scripts\activate
# Using conda
conda create -n pipeframe python=3.10
conda activate pipeframe# Install in editable mode with all dev dependencies
pip install -e ".[dev,test,all]"
# Or from requirements
pip install -r requirements-dev.txtpre-commit install# All tests
pytest
# With coverage
pytest --cov=pipeframe --cov-report=html
# Specific test
pytest tests/test_basic.py -v# Format code
black pipeframe/
isort pipeframe/
# Lint
flake8 pipeframe/
# Type check
mypy pipeframe/# Install build tools
pip install build twine
# Build distribution packages
python -m build
# This creates:
# - dist/pipeframe-0.2.0-py3-none-any.whl
# - dist/pipeframe-0.2.0.tar.gz# Install in clean environment
python -m venv test_env
source test_env/bin/activate
pip install dist/pipeframe-0.2.0-py3-none-any.whl
# Test it works
python -c "from pipeframe import DataFrame; print('Success!')"-
Create PyPI Account
- Go to https://pypi.org/account/register/
- Verify your email
- Enable 2FA (recommended)
-
Create API Token
- Go to https://pypi.org/manage/account/
- Scroll to API tokens section
- Create new token with scope: "Entire account"
- Save the token securely!
-
Configure Token
# Create/edit ~/.pypirc [pypi] username = __token__ password = pypi-YOUR_TOKEN_HERE
# Register on TestPyPI
# https://test.pypi.org/account/register/
# Upload to TestPyPI
twine upload --repository testpypi dist/*
# Test install from TestPyPI
pip install --index-url https://test.pypi.org/simple/ pipeframe# Upload to PyPI
twine upload dist/*
# Verify
pip install pipeframeBefore publishing, ensure:
- Version number updated in
pyproject.tomlandpipeframe/__init__.py - CHANGELOG.md updated with release notes
- All tests passing:
pytest - Code formatted:
black pipeframe/andisort pipeframe/ - Documentation updated
- README.md reviewed
- LICENSE file present
- Git tagged:
git tag v0.2.0 && git push --tags
PipeFrame uses semantic versioning (MAJOR.MINOR.PATCH):
- MAJOR (0.x.x → 1.0.0): Breaking changes
- MINOR (0.2.x → 0.3.0): New features, backward compatible
- PATCH (0.2.0 → 0.2.1): Bug fixes
-
Update version in files:
# pipeframe/__init__.py __version__ = "0.3.0"
# pyproject.toml version = "0.3.0"
-
Update CHANGELOG.md:
## [0.3.0] - 2024-XX-XX ### Added - New feature description
-
Commit and tag:
git add . git commit -m "Bump version to 0.3.0" git tag v0.3.0 git push && git push --tags
- Go to your GitHub repo → Settings → Secrets and variables → Actions
- Add new secret:
PYPI_API_TOKENwith your PyPI token
The package is automatically published when you create a GitHub release:
# Create release on GitHub
# Or use GitHub CLI:
gh release create v0.3.0 \
--title "PipeFrame v0.3.0" \
--notes "Release notes here"The GitHub Action (.github/workflows/publish.yml) will:
- Run tests
- Build the package
- Upload to PyPI
# Install docs dependencies
pip install -e ".[docs]"
# Build docs
cd docs
make html
# View docs
open _build/html/index.htmlDocumentation can be hosted on:
- Read the Docs: https://readthedocs.org (recommended)
- GitHub Pages: Use
gh-pagesbranch - GitLab Pages: Use
.gitlab-ci.yml
# All tests
pytest
# With coverage
pytest --cov=pipeframe --cov-report=html
# View coverage: open htmlcov/index.html
# Specific module
pytest tests/test_basic.py
# Specific test
pytest tests/test_basic.py::TestDataFrame::test_create_dataframe
# Verbose output
pytest -v
# Stop on first failure
pytest -x
# Run in parallel
pytest -n auto# tests/test_feature.py
import pytest
from pipeframe import DataFrame, filter
class TestNewFeature:
def test_feature_works(self):
"""Test the feature works correctly."""
df = DataFrame({'x': [1, 2, 3]})
result = df >> new_feature()
assert len(result) > 0
def test_feature_error(self):
"""Test error handling."""
df = DataFrame({'x': [1, 2, 3]})
with pytest.raises(ValueError):
df >> new_feature(invalid_param=True)Issue: pip install fails
# Upgrade pip
pip install --upgrade pip setuptools wheelIssue: Import errors after install
# Reinstall in editable mode
pip uninstall pipeframe
pip install -e .Issue: Tests fail on import
# Install test dependencies
pip install -e ".[test]"Issue: Version conflicts
# Create fresh environment
python -m venv clean_env
source clean_env/bin/activate
pip install pipeframe- Issues: https://github.com/Yasser03/pipeframe/issues
- Discussions: https://github.com/Yasser03/pipeframe/discussions
- Email: yasser.mustafan@gmail.com
# Development workflow
git clone https://github.com/Yasser03/pipeframe.git
cd pipeframe
python -m venv venv && source venv/bin/activate
pip install -e ".[dev,test]"
pytest # Run tests
black pipeframe/ && isort pipeframe/ # Format
# Publishing workflow
# 1. Update version in pyproject.toml and __init__.py
# 2. Update CHANGELOG.md
# 3. Commit and tag
git commit -am "Release v0.3.0"
git tag v0.3.0
# 4. Build
python -m build
# 5. Publish
twine upload dist/*
# 6. Push
git push && git push --tagsMaintained by Dr. Yasser Mustafa
Built with ❤️ for the data science community