You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/intelligentapps/profiling.md
+35-16Lines changed: 35 additions & 16 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -5,9 +5,9 @@ MetaDescription: Profiling Quickstart in AI Toolkit.
5
5
---
6
6
# Profiling an app using Windows Machine Learning
7
7
8
-
Profiling is a tool designed to help developers and AI engineers to diagnose the CPU, GPU, NPU resource usages of processes, ONNX model on different execution providers, and Windows ML events.
8
+
Profiling is a tool designed to help developers and AI engineers to diagnose CPU, GPU, and NPU resource usage of processes, profile ONNX models on different execution providers, and capture Windows ML events.
9
9
10
-
In this article, you could learn how to start profiling and how to inspect the resource usages view and the events view.
10
+
In this article, you learn how to start profiling and how to inspect the resource usage view and the events view.
11
11
12
12
## Prerequisites
13
13
@@ -17,9 +17,9 @@ In this article, you could learn how to start profiling and how to inspect the r
17
17
## Profile on app startup
18
18
19
19
In this mode, the profiling tool profiles the next app that is started and that is sending out Windows ML events.
20
-
This option is ideal for testing a run-once app. In this case, you start profiling, then run the app, and the resource usages will begin showing up.
20
+
This option is ideal for testing a run-once app. In this case, you start profiling, then run the app, and the resource usage will appear.
21
21
22
-
22
+
23
23
24
24
The tool starts profiling a newly started app. This means that for profiling a Python notebook, if the kernel is already running, you need to restart the kernel to begin profiling for it. Just starting a new notebook does not automatically start profiling.
25
25
@@ -38,15 +38,15 @@ In this mode, the profiling tool starts profiling an already running app. You ca
38
38
39
39
This option is ideal for profiling an app that is already running and you're unable to restart it for profiling purposes.
40
40
41
-
41
+
42
42
43
43
## Profile an ONNX model
44
44
45
-
In this mode, the profiling tool starts profiling an ONNX model file on a target execution provider (EP) or device policy for a given duration. You can see the resource usage while it's running.
45
+
In this mode, the profiling tool starts profiling an ONNX model file on a target execution provider (EP) for a given duration. You can see the resource usage while it's running.
46
46
47
-
This option is ideal for profiling an ONNX model on different EPs or device policies.
47
+
This option is ideal for profiling an ONNX model on different EPs.
48
48
49
-
49
+
50
50
51
51
After profiling, a notification shows up to guide you to open or save the report.
52
52
@@ -56,28 +56,47 @@ The report contains detailed profiling statistics and results for the ONNX model
56
56
57
57
58
58
59
-
##Resource Usages view
59
+
### Benchmark time for each operation
60
60
61
-
In the main window, the plot on the top shows usage of CPU, GPU, NPU, and memory. The usage is updated every second, and kept for 10 minutes. You can use the tools on the top right to navigate the timeline by zooming in, zooming out, and panning.
61
+
If OP Profiling is enabled, op level data will be generated to allow you to inspect the model in more detail.
62
62
63
-
63
+
64
+
65
+
The report contains detailed latencies for each op.
66
+
67
+
68
+
69
+
## Profile an ONNX GenAI model
70
+
71
+
In this mode, the profiling tool starts profiling an ONNX GenAI model on a target execution provider (EP) for a specified number of prompts. You can see the resource usage while it's running.
72
+
73
+
74
+
75
+
> [!NOTE]
76
+
> You need to select the folder of the GenAI model, this is the folder that contains `genai_config.json`.
77
+
78
+
## Resource Usage view
79
+
80
+
In the main window, the plot on the top shows usage of CPU, GPU, NPU, and memory. The usage is updated every second, and is kept for 10 minutes. You can use the tools on the top right to navigate the timeline by zooming in, zooming out, and panning.
81
+
82
+
64
83
65
84
> [!NOTE]
66
85
> This feature uses performance counters. To achieve higher accuracy, you could also try [Windows Performance Recorder](https://learn.microsoft.com/en-us/windows-hardware/test/wpt/windows-performance-recorder).
67
86
68
87
## Windows ML Events view
69
88
70
-
In the main window, the plot on the bottom shows Windows ML events. Its timeline is synced with the Resource Usages view, so you can easily determine how resources are used when certain events occur.
89
+
In the main window, the plot on the bottom shows Windows ML events. Its timeline is synchronized with the Resource Usage view, so you can easily determine how resources are used when certain events occur.
71
90
72
-
> [!Important]
91
+
> [!IMPORTANT]
73
92
> To receive Windows ML events, the tool needs to be run in admin mode. If VS Code is not started in admin mode, a notification shows up and guides you to restart VS Code. You need to close all other VS Code instances to make the restart in admin mode work.
74
93
> 
75
94
76
-
Currently, we only show events of the following types:
95
+
Currently, only the following event types are shown:
77
96
78
97
- Ensure ExecutionProvider Ready: when Windows ML is preparing the EP
79
98
- Session Creation: when the session is created
80
-
- Inference: when the model inferences on the session
99
+
- Inference: when the model runs inference on the session
81
100
82
101
83
102
@@ -86,7 +105,7 @@ Currently, we only show events of the following types:
0 commit comments