-
Notifications
You must be signed in to change notification settings - Fork 179
Feature: Implement Garbage Collection (GC) Management Commands #624
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
intojhanurag
wants to merge
29
commits into
goharbor:main
Choose a base branch
from
intojhanurag:feat/gc-commmand-implementation
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
29 commits
Select commit
Hold shift + click to select a range
aa5fda9
gc commands structure created
intojhanurag 14e4f52
fix:convert dry run coloumn in boolean
intojhanurag 108d0eb
minor_fixing
intojhanurag bca47a1
docs(gc): generate GC command documentation
intojhanurag 46c68e2
fix:Add copywrite headers in all file
intojhanurag c7293c9
feat: Add interactive prompt in update_schedule
intojhanurag 4fa1ae6
feat: Add stop command for running gc
intojhanurag 506d3f5
feat: add docs by dagger
intojhanurag 682ef12
All edge cases covered
intojhanurag 372b2e0
feat: Add pagination in list command
intojhanurag 60100e0
fix: bug in the codebase
intojhanurag 1d3eeec
fix: update_schedule custom command not working
intojhanurag 3df4e6f
add test file for upate_schedule_test
intojhanurag 0536d22
fix: address review feedback on GC commands
intojhanurag 2aee69d
fix: added docs
intojhanurag 6c28fdb
fix: add missing GC query keys and interactive selection for gc log c…
intojhanurag c1d923f
fix:add docs
intojhanurag 4898130
feat: add output-format support for gc list command
intojhanurag fcccab7
refactor: migrate gc selection views to base/selection model and prom…
intojhanurag 996abb6
fix: address review feedback for gc commands
intojhanurag 7eec9cf
fix: use server-side filtering for running GC jobs in stop command
intojhanurag 98620f7
fix: remove committed binary and return error from ListGC instead of …
intojhanurag 3fa2067
fix: Apply minor fixes and bugs
intojhanurag 766f0d0
fix: use correct API query key 'status' instead of 'job_status' for G…
intojhanurag 281126b
fix: add cron expressions for predefined GC schedule types (hourly/…
intojhanurag 702938b
fix: regenerated docs
intojhanurag f25fb36
fix: Apply feedback
intojhanurag 84518a0
fix: clean up gc whitespace
intojhanurag 7e1b88b
docs: regenerate gc command docs
intojhanurag File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,38 @@ | ||
| // Copyright Project Harbor Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package gc | ||
|
|
||
| import ( | ||
| "github.com/spf13/cobra" | ||
| ) | ||
|
|
||
| func GC() *cobra.Command { | ||
| cmd := &cobra.Command{ | ||
| Use: "gc", | ||
| Short: "Manage Garbage Collection", | ||
| Long: "Manage Garbage Collection in Harbor (schedule, history, logs)", | ||
| } | ||
|
|
||
| cmd.AddCommand( | ||
| ListGCCommand(), | ||
| GetGCLogCommand(), | ||
| ViewGCScheduleCommand(), | ||
| UpdateGCScheduleCommand(), | ||
| RunGCCommand(), | ||
| StopGCCommand(), | ||
| ) | ||
|
|
||
| return cmd | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,128 @@ | ||
| // Copyright Project Harbor Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package gc | ||
|
|
||
| import ( | ||
| "fmt" | ||
|
|
||
| "github.com/goharbor/harbor-cli/pkg/api" | ||
| "github.com/goharbor/harbor-cli/pkg/utils" | ||
| "github.com/goharbor/harbor-cli/pkg/views/gc" | ||
| log "github.com/sirupsen/logrus" | ||
| "github.com/spf13/cobra" | ||
| "github.com/spf13/viper" | ||
| ) | ||
|
|
||
| var validGCSortFields = []string{ | ||
| "creation_time", | ||
| "update_time", | ||
| "id", | ||
| "job_status", | ||
| } | ||
|
|
||
| var validGCQueryKeys = []string{ | ||
| "id", | ||
| "status", | ||
| } | ||
|
|
||
| func ListGCCommand() *cobra.Command { | ||
| var ( | ||
| opts api.ListFlags | ||
| sort []string | ||
| fuzzy []string | ||
| match []string | ||
| ranges []string | ||
| ) | ||
|
|
||
| cmd := &cobra.Command{ | ||
| Use: "list", | ||
| Short: "List GC history", | ||
| Long: `List GC (Garbage Collection) history in Harbor. | ||
|
intojhanurag marked this conversation as resolved.
|
||
|
|
||
| This command displays a list of GC executions with their status, creation time, | ||
| and other details. You can control the output using pagination flags and format options. | ||
|
|
||
| Examples: | ||
| # List GC history with default pagination (page 1, 10 items per page) | ||
| harbor gc list | ||
|
|
||
| # List GC history with custom pagination | ||
| harbor gc list --page 2 --page-size 20 | ||
|
|
||
| # List GC history with sorting by creation time (newest first) | ||
| harbor gc list --sort -creation_time | ||
|
|
||
| # List GC history with multiple sort fields | ||
| harbor gc list --sort creation_time --sort -update_time | ||
|
|
||
| # Filter GC history by status (exact match) | ||
| harbor gc list --match status=Success`, | ||
| Args: cobra.ExactArgs(0), | ||
| RunE: func(cmd *cobra.Command, args []string) error { | ||
| if opts.PageSize > 100 { | ||
| return fmt.Errorf("page size should be less than or equal to 100") | ||
| } | ||
|
|
||
| if len(sort) > 0 { | ||
| sortParam, err := utils.BuildSortParam(sort, validGCSortFields) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| opts.Sort = sortParam | ||
| } | ||
|
|
||
| if len(fuzzy) != 0 || len(match) != 0 || len(ranges) != 0 { | ||
| q, qErr := utils.BuildQueryParam(fuzzy, match, ranges, validGCQueryKeys) | ||
| if qErr != nil { | ||
| return qErr | ||
| } | ||
| opts.Q = q | ||
| } | ||
|
|
||
| history, err := api.GetGCHistory(opts) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to get GC history: %v", utils.ParseHarborErrorMsg(err)) | ||
| } | ||
|
|
||
| if len(history) == 0 { | ||
| log.Info("No GC history found") | ||
| return nil | ||
| } | ||
|
|
||
| formatFlag := viper.GetString("output-format") | ||
| if formatFlag != "" { | ||
| err = utils.PrintFormat(history, formatFlag) | ||
| if err != nil { | ||
| return err | ||
| } | ||
| } else { | ||
| if err := gc.ListGC(history); err != nil { | ||
| return err | ||
| } | ||
| } | ||
| return nil | ||
| }, | ||
| } | ||
|
|
||
| flags := cmd.Flags() | ||
| flags.Int64VarP(&opts.Page, "page", "p", 1, "Page number") | ||
| flags.Int64VarP(&opts.PageSize, "page-size", "s", 10, "Size of per page") | ||
| flags.StringSliceVar(&sort, "sort", nil, "Sort the resource list (e.g. --sort creation_time --sort -update_time)") | ||
| flags.StringSliceVar(&fuzzy, "fuzzy", nil, "Fuzzy match filter (key=value)") | ||
| flags.StringSliceVar(&match, "match", nil, "Exact match filter (key=value)") | ||
| flags.StringSliceVar(&ranges, "range", nil, "Range filter (key=min~max)") | ||
|
|
||
| return cmd | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,70 @@ | ||
| // Copyright Project Harbor Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package gc | ||
|
|
||
| import ( | ||
| "fmt" | ||
|
|
||
| "github.com/goharbor/harbor-cli/pkg/api" | ||
| "github.com/goharbor/harbor-cli/pkg/prompt" | ||
| "github.com/spf13/cobra" | ||
| ) | ||
|
|
||
| func GetGCLogCommand() *cobra.Command { | ||
| var gcID int64 | ||
|
|
||
| cmd := &cobra.Command{ | ||
| Use: "log", | ||
| Short: "Get GC job log", | ||
| Long: `Get the log of a specific GC (Garbage Collection) job. | ||
|
|
||
| If no GC job ID is provided via the --id flag, an interactive selector | ||
| will be displayed to choose from available GC jobs. | ||
|
|
||
| Examples: | ||
| # Get GC log by specifying the job ID | ||
| harbor gc log --id 42 | ||
|
|
||
| # Get GC log interactively (select from list) | ||
| harbor gc log`, | ||
| Args: cobra.ExactArgs(0), | ||
| RunE: func(cmd *cobra.Command, args []string) error { | ||
| var err error | ||
|
|
||
| if gcID < 0 { | ||
| return fmt.Errorf("invalid GC job ID: %d. ID must be a positive number", gcID) | ||
| } | ||
|
|
||
| if gcID == 0 { | ||
| gcID, err = prompt.GetGCJobIDFromUser() | ||
| if err != nil { | ||
| return err | ||
| } | ||
| } | ||
|
|
||
| logData, err := api.GetGCJobLog(gcID) | ||
| if err != nil { | ||
| return fmt.Errorf("failed to get GC log: %v", err) | ||
| } | ||
|
|
||
| fmt.Println(logData) | ||
| return nil | ||
| }, | ||
| } | ||
|
|
||
| cmd.Flags().Int64Var(&gcID, "id", 0, "ID of the GC job to get logs for") | ||
|
|
||
| return cmd | ||
| } | ||
|
intojhanurag marked this conversation as resolved.
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| // Copyright Project Harbor Authors | ||
| // | ||
| // Licensed under the Apache License, Version 2.0 (the "License"); | ||
| // you may not use this file except in compliance with the License. | ||
| // You may obtain a copy of the License at | ||
| // | ||
| // http://www.apache.org/licenses/LICENSE-2.0 | ||
| // | ||
| // Unless required by applicable law or agreed to in writing, software | ||
| // distributed under the License is distributed on an "AS IS" BASIS, | ||
| // WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| // See the License for the specific language governing permissions and | ||
| // limitations under the License. | ||
|
|
||
| package gc | ||
|
|
||
| import ( | ||
| "fmt" | ||
|
|
||
| "github.com/goharbor/go-client/pkg/sdk/v2.0/models" | ||
| "github.com/goharbor/harbor-cli/pkg/api" | ||
| "github.com/goharbor/harbor-cli/pkg/utils" | ||
| log "github.com/sirupsen/logrus" | ||
| "github.com/spf13/cobra" | ||
| ) | ||
|
|
||
| func RunGCCommand() *cobra.Command { | ||
| var dryRun, deleteUntagged bool | ||
| var workers int | ||
|
|
||
| cmd := &cobra.Command{ | ||
| Use: "run", | ||
| Short: "Run Garbage Collection manually", | ||
|
intojhanurag marked this conversation as resolved.
|
||
| Args: cobra.ExactArgs(0), | ||
| RunE: func(cmd *cobra.Command, args []string) error { | ||
| scheduleObj := models.ScheduleObj{ | ||
| Type: "Manual", | ||
| } | ||
|
|
||
| params := map[string]interface{}{ | ||
| "dry_run": dryRun, | ||
| "delete_untagged": deleteUntagged, | ||
| "workers": workers, | ||
| } | ||
|
|
||
| scheduleBody := &models.Schedule{ | ||
| Schedule: &scheduleObj, | ||
| Parameters: params, | ||
| } | ||
|
|
||
| err := api.CreateGCSchedule(scheduleBody) | ||
|
intojhanurag marked this conversation as resolved.
intojhanurag marked this conversation as resolved.
|
||
| if err != nil { | ||
| return fmt.Errorf("failed to start GC: %v", utils.ParseHarborErrorMsg(err)) | ||
| } | ||
| log.Info("GC started successfully") | ||
| return nil | ||
| }, | ||
| } | ||
|
|
||
| cmd.Flags().BoolVarP(&dryRun, "dry-run", "", false, "Simulate GC without deleting artifacts") | ||
| cmd.Flags().BoolVarP(&deleteUntagged, "delete-untagged", "", true, "Delete untagged artifacts") | ||
| cmd.Flags().IntVar(&workers, "workers", 1, "Number of workers for GC job") | ||
|
|
||
| return cmd | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.