only check for delegation token from NN when it is non S3 filesystem - #50
Draft
johnnysohn wants to merge 51 commits into
Draft
only check for delegation token from NN when it is non S3 filesystem#50johnnysohn wants to merge 51 commits into
johnnysohn wants to merge 51 commits into
Conversation
… of invalidated blocks
* Build Hadoop for arm64 * Rely on buildpack for docker usage * Use Java 11 * try without maven cache * Compute env vars inside build.sh * use buildpack maven support * back to master dependencies Co-authored-by: Charles Connell <cconnell@hubspot.com>
Co-authored-by: Charles Connell <cconnell@hubspot.com>
Co-authored-by: Charles Connell <cconnell@hubspot.com>
Co-authored-by: Charles Connell <cconnell@hubspot.com>
Co-authored-by: Charles Connell <cconnell@hubspot.com>
Co-authored-by: Charles Connell <cconnell@hubspot.com>
Co-authored-by: Charles Connell <cconnell@hubspot.com>
Co-authored-by: Charles Connell <cconnell@hubspot.com>
Co-authored-by: Charles Connell <cconnell@hubspot.com>
…fallback (#26) Co-authored-by: Charles Connell <cconnell@hubspot.com>
Hadoop release process involves creating a new branch for every release, so we can't easily follow the expected process of having a branch per major release. Instead we need to have a branch per release as well, and cherry-pick our changes over to each new release. Update our build script and rpm spec to account.
Co-authored-by: Charles Connell <cconnell@hubspot.com>
* Update .blazar.yaml * Update .blazar.yaml * Update .blazar.yaml * Update .blazar.yaml
- Update build versioning to be based around patch versions - Use single-module builds to ease dependency graph issues - Force HBase1 building to work around activeByDefault behavior
…ble with node.js 12 to avoid compilation error. (apache#5827). Contributed by Masatake Iwasaki Reviewed-by: Shilun Fan <slfan1989@apache.org> Signed-off-by: Ayush Saxena <ayushsaxena@apache.org> (cherry picked from commit a822a3c)
* Persist build env variables withour build rc file * Add clarifying comment * Cleanup HADOOP_VERSION environment variable * Copy changes to all blazar yaml files * Revert "Copy changes to all blazar yaml files" This reverts commit 5f4457e. --------- Co-authored-by: Samuel McCann <smccann@hubspot.com>
…ing migration to 3.3.6
…when datanodes close idle connections
…ing dn with many ec blocks (apache#4398)
…y max-streams-hard-limit configuration (apache#5143) Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
…freshable (apache#4567) Reviewed-by: Hiroyuki Adachi <hadachi@yahoo-corp.jp> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
…plicatedTasks and numReplicationTasks during block recovery (apache#6348) Reviewed-by: Shilun Fan <slfan1989@apache.org> Signed-off-by: Takanobu Asanuma <tasanuma@apache.org>
…native codec. (apache#5520). Contributed by WangYuanben. Reviewed-by: Tao Li <tomscut@apache.org> Reviewed-by: Shilun Fan <slfan1989@apache.org> Signed-off-by: Ayush Saxena <ayushsaxena@apache.org>
* Remove deployAtEnd because it disables publishing * Increase build timeout for Hadoop --------- Co-authored-by: Derik DeLong <ddelong@hubspot.com>
…ddTransferRateMetric(). (apache#6290). Contributed by Xing Lin. Reviewed-by: Ravindra Dingankar <rdingankar@linkedin.com> Reviewed-by: Simbarashe Dzinamarira <sdzinamarira@linkedin.com> Reviewed-by: Tao Li <tomscut@apache.org> Signed-off-by: He Xiaoqiao <hexiaoqiao@apache.org>
- Change build order to avoid deploy plugin defect - Remove build exclusion - Restore original location for argument, remove weird skip config - Move the deployAtEnd to a different spot
- Leave config in same place - Disable deployAtEnd - Increase build timeout for Hadoop
* Use new rpm buildpack * Update .blazar.yaml --------- Co-authored-by: Scott Williams <srwilliams@hubspot.com>
Co-authored-by: Charles Connell <cconnell@hubspot.com>
charlesconnell
force-pushed
the
hubspot-3.3.6
branch
2 times, most recently
from
July 29, 2026 16:28
0057a70 to
c1f38bb
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description of PR
We have an issue where writing to S3 fails because we're checking delegation token before writing in FileOutputFormat. Normally call to
TokenCache.obtainTokensForNamenodesshouldn't fail in this case of writing to S3 because it returns if security isn't enabled, but we have jobs that can do lots of things and within it also makes calls to HBase to authenticate and thus creates problem for FileOutputFormat writing to S3.How was this patch tested?
For code changes:
LICENSE,LICENSE-binary,NOTICE-binaryfiles?