Torchserve 23mt v0.8.0#13
Open
simonschoelly wants to merge 174 commits intotorchserve-23mt-v0.7.1from
Open
Conversation
Co-authored-by: Mark Saroufim <marksaroufim@fb.com>
Co-authored-by: Mark Saroufim <marksaroufim@fb.com>
Co-authored-by: Mark Saroufim <marksaroufim@fb.com>
…2118) * Added an example serving models using TorchServe Docker Container * Updated examples README with a link to the new readme * added link to example in docker readme
* Create codeql
…ytorch#2154) * Changes to support different python versions * Updated default version to 3.9 * spellcheck lint
* Update model file docs to be more accurate * model-archiver: Improve docs --------- Co-authored-by: Ankith Gunapal <agunapal@meta.com>
…nts using glob (pytorch#2142) Co-authored-by: Ankith Gunapal <agunapal@meta.com> Co-authored-by: Mark Saroufim <marksaroufim@fb.com>
Co-authored-by: Matthias Reso <13337103+mreso@users.noreply.github.com>
* Upload benchmark artifacts for auto-validation
* Make torch_handler tests callable with pytest * Add handler test to sanity check * Replace resnet 152 against 18 to speedup unit test; also fixes issue with old weights version on gpu * Fix lint error
* feat: nvida dali torchserve integration Signed-off-by: jagadeeshi2i <jagadeeshj@ideas2it.com> * typos and naming changes in dali integration * fix: change env var to file check in vision handler Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix: dali imports Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix: set prefetch queue depth as constant Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix: vision handler Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix: readme doc Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix: spellcheck Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix: lint error Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix: serializer script Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * Added unit test for dali integration * fix: lint errors Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix typo Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix: move dali preprocessor to custom handler Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> * fix: unit test Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> --------- Signed-off-by: jagadeeshi2i <jagadeeshj@ideas2it.com> Signed-off-by: jagadeesh <jagadeeshj@ideas2it.com> Co-authored-by: Mark Saroufim <marksaroufim@fb.com>
* BERT nightly benchmark on Inferentia1 * Consolidate neuron benchmark model config files into a single file for BERT Set the NEURON_RT_NUM_CORES value as a string in the inf1 nightly benchmark workflow file * Update trnsformer model downloader documentation * test workflow before merge * Consolidate benchmark workflows * Update runs-on syntax * Remove hardware specific benchmark workflow files * Consolidate install dependencies step Add necessary env variables for cpu and inf1 Disable fail-fast to enable all benchmarks to run even if one of them fail * update documentation --------- Co-authored-by: Naman Nandan <namannan@amazon.com>
* Make model-archiver tests independent from caller directory * Make model-archiver tests callable from any directory
* Support for PyTorch 2.0 * Ignoring DALI pytest * Updateregression tests to run weekly
Co-authored-by: Naman Nandan <namankt55@gmail.com> Co-authored-by: Ankith Gunapal <agunapal@meta.com>
* feat:grpc server side streaming * add regression test for grpc stream * add word in lint spell list * fmt * fmt * update doc * add LLMs in wordlist * update doc
…ytorch#2154) * Changes to support different python versions * Updated default version to 3.9 * spellcheck lint
Co-authored-by: Ankith Gunapal <agunapal@ischool.Berkeley.edu>
pytorch#2371) Co-authored-by: Naman Nandan <namannan@amazon.com>
* Enable ONNX test in CI
* update cpuinfo * update * lint * lint * lint * install numactl --------- Co-authored-by: Ankith Gunapal <agunapal@ischool.Berkeley.edu> Co-authored-by: Mark Saroufim <marksaroufim@fb.com>
* fix tcp port in model_service_worker.py * update torchrun parameter name * fmt
* update cpuinfo * update * lint * lint * lint * install numactl * update num workers * skip if gpu --------- Co-authored-by: Ankith Gunapal <agunapal@ischool.Berkeley.edu> Co-authored-by: Mark Saroufim <marksaroufim@fb.com>
…h#2374) torch == 2.0.1 torchvision == 0.15.2 torchtext == 0.15.2 torchaudio == 2.0.2 Co-authored-by: Naman Nandan <namannan@amazon.com> Co-authored-by: Ankith Gunapal <agunapal@ischool.Berkeley.edu>
* adding type annotations to functions
* reflect the model config from properties * honor model config in config.properties
Bumps [transformers](https://github.com/huggingface/transformers) from 4.28.1 to 4.30.0. - [Release notes](https://github.com/huggingface/transformers/releases) - [Commits](huggingface/transformers@v4.28.1...v4.30.0) --- updated-dependencies: - dependency-name: transformers dependency-type: direct:development ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…ytorch#2350) * add jobQsize * add jobQueueSize in model_config.yaml for mnist_scripted.mar * feature job tickets * rename hasJobTickets to getJobTickets * fmt doc * update doc * fmt doc * fmt doc * update doc
…:pytorch/serve into torchserve-23mt-v0.8.0-fix-1
Torchserve 23mt v0.8.0 fix 1
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
17 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR rebases our changes on v0.8.0 of torch-serve
This PR shoud not be merged into
based-on-v0.8.0- I just made this a base branch here, so that one can see the changes during code review.