Skip to content

Resolve bitBucket 7.x Issue #22

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ywei2017 wants to merge 4 commits into
base: master
Choose a base branch
from
Open

Resolve bitBucket 7.x Issue #22

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
9bdea4a
Use bitBucket API to get the source/target commits, as the
ywei2017 May 9, 2022
0cecdfa
Install git-lfs from apk
ywei2017 May 9, 2022
ff74f39
set merge branch consistently between check and in
ywei2017 May 9, 2022
3ff0891
Added max_pr_to_check to improve efficiency
ywei2017 May 10, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,16 @@ RUN apk --no-cache add \
ca-certificates=20190108-r0 \
curl=7.61.1-r3 \
git=2.15.4-r0 \
git-lfs>1.1.2 \
jq=1.5-r5 \
openssh-client=7.5_p1-r10

# can't `git pull` unless we set these
RUN git config --global user.email "git@localhost" && \
git config --global user.name "git"

COPY scripts/install_git_lfs.sh install_git_lfs.sh
RUN ./install_git_lfs.sh
# No longer needed as git-lfs is installed with apk
#COPY scripts/install_git_lfs.sh install_git_lfs.sh
#RUN ./install_git_lfs.sh

COPY assets /opt/resource
2 changes: 2 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,8 @@ It will accept a regular expression as determined by [egrep](http://linuxcommand

* `only_when_mergeable`: *Optional (default: false).* If enabled only pull requests which are mergeable (all tasks done, required number of approvers reached, ...) will be built.

* `max_pr_to_check`: *Optional (default: 999).* Max number of PRs to check, ordered in descending order by PR number.
Copy link
Copy Markdown
Author

@ywei2017 ywei2017 May 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Our corporate bitBucket server is pretty slow, and heavily loaded. And it has days REALLY, REALLY slow, so after much hesitation I added this flag. For most people, nothing needs to be done, and a default of 999 is plenty enough. For the rest of us, giving it a limit speeds things up quite a bit.


* `only_when_asked`: *Optional (default: false).* Only build pull request when explicitly asked for, using rebuild_phrase.

* `rebuild_when_target_changed`: *Optional (default: false).* Rebuild pull requests when target branch changed instead of only when source branch changed.
Expand Down
4 changes: 3 additions & 1 deletion assets/check
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ sleep_between_fetches=$(jq -r '.source.sleep_between_fetches // "0"' < "$payload
sleep_between_fetches=$(echo "$sleep_between_fetches" | sed 's/[^0-9\.]//g')
rebuild_when_target_changed=$(jq -r '.source.rebuild_when_target_changed // "false"' < "$payload")
rebuild_phrase=$(jq -r '.source.rebuild_phrase // "test this please"' < "$payload")
max_pr_to_check=$(jq -r '.source.max_pr_to_check // "999"' < "$payload")
CURRENT_VERSION_DATE=$(jq -r '.version.date // "0"' < "$payload")

configure_git_ssl_verification "$skip_ssl_verification"
Expand All @@ -60,7 +61,8 @@ fi
# collect all pull requests from uri
REMOTES=$(git ls-remote "$uri")
set +e
PULL_REQUESTS=$(echo "$REMOTES" | grep -E "/pull\\-requests/[0-9]+/${prq_branch}")
PULL_REQUESTS=$(echo "$REMOTES" | grep -E "/pull\\-requests/[0-9]+/${prq_branch}" | \
sort -t / -k 3 -n -r | head -n $max_pr_to_check)
Copy link
Copy Markdown
Author

@ywei2017 ywei2017 May 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sort by the PR number and sort in reverse, taking the top $max_pr_to_check ensures we get the most recent ones.

set -e


Expand Down
11 changes: 7 additions & 4 deletions assets/helpers/git.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -96,14 +96,17 @@ pullrequest_metadata() {
# $1: pull request number
# $2: pull request repository
# $3: skip ssl verification
# $4: soucre_commmit
# $5: target_commmit

local source_commit=$(git rev-list --parents -1 $(git rev-parse HEAD) | awk '{print $3}')
local target_commit=$(git rev-list --parents -1 $(git rev-parse HEAD) | awk '{print $2}')
# No longer working with vSphere 7
#local source_commit=$(git rev-list --parents -1 $(git rev-parse HEAD) | awk '{print $3}')
#local target_commit=$(git rev-list --parents -1 $(git rev-parse HEAD) | awk '{print $2}')

jq -n "[]" | \
add_pullrequest_metadata_basic "$1" "$2" "$3" | \
add_pullrequest_metadata_commit "source" "$source_commit" | \
add_pullrequest_metadata_commit "target" "$target_commit"
add_pullrequest_metadata_commit "source" "$4" | \
add_pullrequest_metadata_commit "target" "$5"
}

configure_credentials() {
Expand Down
45 changes: 27 additions & 18 deletions assets/in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,13 @@ if test "$depth" -gt 0 2> /dev/null; then
depthflag="--depth $depth"
fi

branch="pull-requests/${prq_id}/merge"
# see comments in "check" resource
branch="pull-requests/${prq_id}"
if [ "$rebuild_when_target_changed" == "true" ]; then
branch="$branch/merge"
else
branch="$branch/from"
fi

log "Cloning $uri in $destination"
git clone $depthflag "$uri" "$destination"
Expand Down Expand Up @@ -118,20 +124,6 @@ if [ "$disable_git_lfs" != "true" ]; then
git submodule foreach "git lfs fetch && git lfs checkout"
fi

# calculate source and target commit
source_commit=$(git rev-list --parents -1 $ref | awk '{print $3}')
target_commit=$(git rev-list --parents -1 $ref | awk '{print $2}')

if [ -z "$source_commit" ]; then
log "Unable to determine source commit from merge commit $ref. Please verify depth configuration."
exit 1
fi

if [ -z "$target_commit" ]; then
log "Unable to determine target commit from merge commit $ref. Please verify depth configuration."
exit 1
fi

# parse uri and retrieve host
uri_parser "$uri"
repo_host="${uri_schema}://${uri_address}"
Expand All @@ -147,19 +139,36 @@ prq=$(bitbucket_pullrequest "$repo_host" "$repo_project" "$repo_name" "$prq_id"
if [ "$prq" != "NO_SUCH_PULL_REQUEST" ] && \
[ "$prq" != "ALREADY_MERGED" ] && \
[ "$prq" != "DECLINED" ]; then
branch=$(echo "$prq" | jq -r '.fromRef.displayId')
from_branch=$(echo "$prq" | jq -r '.fromRef.displayId')
fi

# calculate source and target commit
Copy link
Copy Markdown
Author

@ywei2017 ywei2017 May 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

get the info from the API, instead of rev-list

source_commit=$(echo "$prq" | jq -r '.fromRef.latestCommit')
target_commit=$(echo "$prq" | jq -r '.toRef.latestCommit')
if [ -z "$source_commit" ]; then
log "Unable to determine source commit from merge commit $ref. Please verify depth configuration."
exit 1
fi

if [ -z "$target_commit" ]; then
log "Unable to determine target commit from merge commit $ref. Please verify depth configuration."
exit 1
fi

if [[ "$branch" == */from ]]; then
Copy link
Copy Markdown
Author

@ywei2017 ywei2017 May 10, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the source branch is /from, we are merging in the changes from the $target_commit so we fake it like /merge'. This is important for the get` steps, which is often followed by the pipeline figuring out "what changed" in the PR? Without this step, the PR will miss the changes on the target branch and show them as altered files.

log "Merge in changes from $target_commit to pullrequest/*/from, to emulate pullrequest/*/merge"
git merge $target_commit
fi

# expose configuration of pull request that can be used in container
git config --add pullrequest.id $prq_id
git config --add pullrequest.source $source_commit
git config --add pullrequest.target $target_commit
git config --add pullrequest.merge $ref
git config --add pullrequest.date "$prq_date"
git config --add pullrequest.branch "$branch"
git config --add pullrequest.branch "$from_branch"

jq -n "{
version: $(jq '.version' < "$payload"),
metadata: $(pullrequest_metadata "$prq_id" "$uri" "$skip_ssl_verification")
metadata: $(pullrequest_metadata "$prq_id" "$uri" "$skip_ssl_verification" "$source_commit" "$target_commit" )
}" >&3
12 changes: 7 additions & 5 deletions assets/out
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,10 +123,6 @@ case "$status" in
exit 1
esac

# set build status on commit in bitbucket, this has to be the latest source commit or the pull request won't pick it up
source_commit=$(git rev-list --parents -1 $merge_commit | awk '{print $3}')
target_commit=$(git rev-list --parents -1 $merge_commit | awk '{print $2}')

# determine repository name for calling REST api
repo_name=$(basename "$uri" | sed "s/.git$//")
repo_project=$(basename $(dirname "$uri"))
Expand All @@ -135,6 +131,12 @@ repo_project=$(basename $(dirname "$uri"))
uri_parser "$uri"
repo_host="${uri_schema}://${uri_address}"$(getBasePathOfBitbucket)

# calculate source and target commit
prq=$(bitbucket_pullrequest "$repo_host" "$repo_project" "$repo_name" "$prq_number" "" "$skip_ssl_verification")
source_commit=$(echo "$prq" | jq -r '.fromRef.latestCommit')
target_commit=$(echo "$prq" | jq -r '.toRef.latestCommit')


# include ATC_EXTERNAL_URL in build status key, different sources should have different builds
build_key="$BUILD_TEAM_NAME-$BUILD_PIPELINE_NAME-$BUILD_JOB_NAME-$ATC_EXTERNAL_URL"
build_name="Concourse $BUILD_TEAM_NAME: $BUILD_PIPELINE_NAME - $BUILD_JOB_NAME - #$BUILD_NAME"
Expand Down Expand Up @@ -189,5 +191,5 @@ jq -n "{
hash: \"$prq_hash\",
date: \"$PULL_REQUEST_DATE\"
},
metadata: $(pullrequest_metadata "$prq_number" "$uri" "$skip_ssl_verification")
metadata: $(pullrequest_metadata "$prq_number" "$uri" "$skip_ssl_verification" "$source_commit" "$target_commit" )
}" >&3
1 change: 1 addition & 0 deletions scripts/install_git_lfs.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
#!/bin/sh
## Why is this needed instead of inistalling from apk?

set -eu

Expand Down