Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 39 additions & 24 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -159,7 +159,17 @@ We offer a comprehensive guide to walk you through the process of optimizing an
<tr>
<td width="30%" align="center">Compress</td>
<td width="30%" align="center">np.compressor</td>
<td width="40%" align="center">Compress and optimize the user’s model.</td>
<td width="40%" align="center">Compress the user’s model.</td>
</tr>
<tr>
<td width="30%" align="center">Optimize</td>
<td width="30%" align="center">np.optimizer</td>
<td width="40%" align="center">Optimize the user’s model.</td>
</tr>
<tr>
<td width="30%" align="center">Simulate</td>
<td width="30%" align="center">np.simulator</td>
<td width="40%" align="center">Simulate the user’s model.</td>
</tr>
<tr>
<td width="30%" align="center">Quantize</td>
Expand All @@ -172,9 +182,9 @@ We offer a comprehensive guide to walk you through the process of optimizing an
<td width="40%" align="center">Convert and quantize the user’s model to run efficiently on device.</td>
</tr>
<tr>
<td width="30%" align="center">Benchmark</td>
<td width="30%" align="center">np.benchmarker</td>
<td width="40%" align="center">Benchmark the user's model to measure model inference speed on diverse device.</td>
<td width="30%" align="center">Profile</td>
<td width="30%" align="center">np.profiler</td>
<td width="40%" align="center">Profile the user's model to measure model inference speed on diverse device.</td>
</tr>
</table>
</div>
Expand Down Expand Up @@ -204,6 +214,11 @@ Log-in to your netspresso account. Please sign-up [here](https://netspresso.ai/?
```python
from netspresso import NetsPresso

# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
```

Expand Down Expand Up @@ -398,11 +413,11 @@ conversion_result = converter.convert_model(
)
```

### Benchmarker
### Profiler

#### Benchmark
#### Profile

To start benchmarking a model, enter the model path to benchmark and the target device name.
To start profiling a model, enter the model path to profile and the target device name.

For NVIDIA GPUs and Jetson devices, device name and software version have to be matched with the target device of the conversion.

Expand All @@ -411,22 +426,22 @@ TensorRT Model has strong dependency with the device type and its jetpack versio
```python
from netspresso.enums import DeviceName, SoftwareVersion

# 1. Declare benchmarker
benchmarker = netspresso.benchmarker_v2()
# 1. Declare profiler
profiler = netspresso.profiler_v2()

# 2. Run benchmark
benchmark_result = benchmarker.benchmark_model(
# 2. Run profile
profile_result = profiler.profile_model(
input_model_path="./outputs/converted/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1/TENSORRT_JETSON_AGX_ORIN_JETPACK_5_0_1.trt",
target_device_name=DeviceName.JETSON_AGX_ORIN,
target_software_version=SoftwareVersion.JETPACK_5_0_1,
)
print(f"model inference latency: {benchmark_result.benchmark_result.latency} ms")
print(f"model gpu memory footprint: {benchmark_result.benchmark_result.memory_footprint_gpu} MB")
print(f"model cpu memory footprint: {benchmark_result.benchmark_result.memory_footprint_cpu} MB")
print(f"model inference latency: {profile_result.profile_result.latency} ms")
print(f"model gpu memory footprint: {profile_result.profile_result.memory_footprint_gpu} MB")
print(f"model cpu memory footprint: {profile_result.profile_result.memory_footprint_cpu} MB")
```

<details open>
<summary>Supported options for Converter & Benchmarker</summary>
<summary>Supported options for Converter & Profier</summary>
<div markdown="1">

### Frameworks that support conversion for model's framework
Expand All @@ -438,7 +453,7 @@ print(f"model cpu memory footprint: {benchmark_result.benchmark_result.memory_fo
| OPENVINO | ✔️ | | |
| TENSORFLOW_LITE | ✔️ | ✔️ | ✔️ |

### Devices that support benchmarks for model's framework
### Devices that support profiles for model's framework

| Device / Framework | ONNX | TENSORRT | TENSORFLOW_LITE | DRPAI | OPENVINO |
|:-----------------------------|:----:|:--------:|:---------------:|:-----:|:--------:|
Expand All @@ -463,7 +478,7 @@ print(f"model cpu memory footprint: {benchmark_result.benchmark_result.memory_fo
| AWS_T4 | ✔️ | ✔️ | | | |
| INTEL_XEON_W_2233 | | | | | ✔️ |

### Software versions that support conversions and benchmarks for specific devices
### Software versions that support conversions and profiles for specific devices

Software Versions requires for Jetson Device. If you are using a different device, you do not need to enter it.

Expand All @@ -485,25 +500,25 @@ print(f"model cpu memory footprint: {benchmark_result.benchmark_result.memory_fo
target_device_name=DeviceName.JETSON_AGX_ORIN,
target_software_version=SoftwareVersion.JETPACK_5_0_1,
)
benchmark_result = benchmarker.benchmark_model(
profile_result = profiler.profile_model(
input_model_path=CONVERTED_MODEL_PATH,
target_device_name=DeviceName.JETSON_AGX_ORIN,
target_software_version=SoftwareVersion.JETPACK_5_0_1,
)
```

### Hardware type that support benchmarks for specific devices
### Hardware type that support profiles for specific devices

Benchmark and compare models with and without Arm Helium.
Profile and compare models with and without Arm Helium.

`RENESAS_RA8D1` and `ALIF_ENSEMBLE_E7_DEVKIT_GEN2` are available for use.

The benchmark results with Helium can be up to twice as fast as without Helium.
The profile results with Helium can be up to twice as fast as without Helium.

The code below is an example of using hardware type.

```python
benchmark_result = benchmarker.benchmark_model(
profile_result = profiler.profile_model(
input_model_path=CONVERTED_MODEL_PATH,
target_device_name=DeviceName.RENESAS_RA8D1,
target_data_type=DataType.INT8,
Expand Down Expand Up @@ -537,8 +552,8 @@ print(f"model cpu memory footprint: {benchmark_result.benchmark_result.memory_fo
<td align="center">50</td>
</tr>
<tr>
<td align="center">Benchmarker</td>
<td align="center">Benchmark</td>
<td align="center">Profiler</td>
<td align="center">Profile</td>
<td align="center">25</td>
</tr>
</table>
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,8 +75,12 @@ FRAMEWORK = Framework.PYTORCH
```python
from netspresso import NetsPresso

# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

compressor = netspresso.compressor_v2()
model = compressor.upload_model(
Expand Down Expand Up @@ -165,8 +169,12 @@ from netspresso import NetsPresso
from netspresso.enums import CompressionMethod, Policy, LayerNorm, GroupPolicy
from netspresso.clients.compressor.v2.schemas import Options

# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

compressor = netspresso.compressor_v2()
compression_info = compressor.select_compression_method(
Expand Down Expand Up @@ -291,8 +299,12 @@ from netspresso import NetsPresso
from netspresso.enums import CompressionMethod, GroupPolicy, LayerNorm, Policy
from netspresso.clients.compressor.v2.schemas import Options

# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

# 1. Declare compressor
compressor = netspresso.compressor_v2()
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,12 @@ from netspresso import NetsPresso
from netspresso.enums import CompressionMethod, RecommendationMethod


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

compressor = netspresso.compressor_v2()
compressed_model = compressor.recommendation_compression(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,12 @@
from netspresso import NetsPresso


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

compressor = netspresso.compressor_v2()
compressed_model = compressor.automatic_compression(
Expand Down
7 changes: 6 additions & 1 deletion docs/description/netspresso/converter/converter.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@ from netspresso import NetsPresso
from netspresso.enums import DeviceName, Framework, SoftwareVersion


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

converter = netspresso.converter_v2()
conversion_task = converter.convert_model(
Expand Down
14 changes: 12 additions & 2 deletions docs/description/netspresso/optimizer/optimizer.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,12 @@ The following pattern handlers are available for graph optimization:
from netspresso import NetsPresso
from netspresso.enums.graph_optimize import GraphOptimizePatternHandler

netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

# Initialize graph optimizer
optimizer = netspresso.graph_optimizer()
Expand All @@ -69,7 +74,12 @@ optimized_model = optimizer.optimize(
from netspresso import NetsPresso
from netspresso.enums.graph_optimize import GraphOptimizePatternHandler

netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

# Initialize graph optimizer
optimizer = netspresso.graph_optimizer()
Expand Down
7 changes: 6 additions & 1 deletion docs/description/netspresso/profiler/profiler.md
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@ from netspresso import NetsPresso
from netspresso.enums import DeviceName, SoftwareVersion


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

profiler = netspresso.profiler()
profiling_task = profiler.profile_model(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@ from netspresso import NetsPresso
from netspresso.enums import QuantizationPrecision


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

quantizer = netspresso.quantizer()
quantization_result = quantizer.automatic_quantization(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@ from netspresso import NetsPresso
from netspresso.enums import QuantizationPrecision


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

quantizer = netspresso.quantizer()

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@ from netspresso import NetsPresso
from netspresso.enums import QuantizationPrecision


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

quantizer = netspresso.quantizer()

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@ from netspresso import NetsPresso
from netspresso.enums import QuantizationPrecision


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

quantizer = netspresso.quantizer()
recommendation_metadata = quantizer.get_recommendation_precision(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,12 @@ from netspresso import NetsPresso
from netspresso.enums import QuantizationPrecision, SimilarityMetric


netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

quantizer = netspresso.quantizer()
quantization_result = quantizer.uniform_precision_quantization(
Expand Down
7 changes: 6 additions & 1 deletion docs/description/netspresso/simulator/simulator.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,12 @@
from netspresso import NetsPresso
from pathlib import Path

netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

# Initialize simulator
simulator = netspresso.simulator()
Expand Down
12 changes: 10 additions & 2 deletions docs/description/netspresso/trainer/trainer.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,12 @@ from netspresso.trainer.augmentations import Resize
from netspresso.trainer.optimizers import AdamW
from netspresso.trainer.schedulers import CosineAnnealingWarmRestartsWithCustomWarmUp

# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

# 1. Declare trainer
trainer = netspresso.trainer(task=Task.OBJECT_DETECTION)
Expand Down Expand Up @@ -69,8 +73,12 @@ training_result = trainer.train(gpus="0, 1", project_name="project_sample")
from netspresso import NetsPresso
from netspresso.trainer.optimizers import AdamW

# Login with API key (recommended)
# Get your API token from: https://account.netspresso.ai/api-token
netspresso = NetsPresso(api_key="YOUR_API_KEY")

netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")
# Note: Email/password login will be deprecated soon
# netspresso = NetsPresso(email="YOUR_EMAIL", password="YOUR_PASSWORD")

# 1. Declare trainer
trainer = netspresso.trainer(yaml_path="./temp/hparams.yaml")
Expand Down