Add bounding box annotations by BaileyJoe · Pull Request #30 · NUbots/NUpbr

BaileyJoe · 2026-01-15T23:11:08Z

Adds bounding box annotations to output in YOLO format.

Too do: fix some issues with field line annotations.

Sylforen · 2026-02-20T09:37:03Z

ISSUE 1: Dependencies weren't installing correctly into Blender's embedded Python directory via the supplied script. It could be because I am running Fedora OS.

SOLVED:
cd -> /blender-2.93.9-linux-x64/2.93/python/bin & run this in BASH

./python3.9 -m ensurepip \
./python3.9 -m pip install -upgrade pip \
./python3.9 -m pip install opencv-python

ISSUE 2: After dependencies were installed and /resources from resources.zip was placed in /NUpbr (root). Blender was attempted to be ran in headless, which produced /meta + /raw + /seg in /outputs/run_1. However, each of the folder's are empty. An error message suggests that /NUpbr/resources/robot/GankenKun.json is missing.

austin@fedora:~/Personal/NUBots/jo_pr_1/NUpbr$ ../blender-2.93.9-linux-x64/blender -P pbr/pbr.py 
Read prefs: /home/austin/.config/blender/2.93/config/userpref.blend
[INFO] Importing balls from '/home/austin/Personal/NUBots/jo_pr_1/NUpbr/resources/balls'
[INFO] 	Number of balls imported: 110
[INFO] Importing environments from '/home/austin/Personal/NUBots/jo_pr_1/NUpbr/resources/hdr'
[INFO] 	Number of environments imported: 3
[INFO] Importing grass textures from '/home/austin/Personal/NUBots/jo_pr_1/NUpbr/resources/grass'
[INFO] 	Number of grass textures imported: 1
FBX Import: start importing /home/austin/Personal/NUBots/jo_pr_1/NUpbr/resources/robot/NUgus_esh.fbx
FBX version: 7400
	FBX import: Prepare...
		Done (0.003456 sec)

	FBX import: Templates...
		Done (0.000957 sec)

	FBX import: Nodes...
		Done (0.001453 sec)

	FBX import: Connections...
		Done (0.001677 sec)

	FBX import: Meshes...
		Done (0.331853 sec)

	FBX import: Materials & Textures...
		Done (0.000044 sec)

	FBX import: Cameras & Lamps...
		Done (0.000116 sec)

	FBX import: Objects & Armatures...
		Done (0.001375 sec)

	FBX import: ShapeKeys...
		Done (0.000018 sec)

	FBX import: Animations...
		Done (0.000025 sec)

	FBX import: Assign materials...
		Done (0.000055 sec)

	FBX import: Assign textures...
		Done (0.000009 sec)

	FBX import: Cycles z-offset workaround...
		Done (0.000007 sec)

	Done (0.778988 sec)

Import finished.
FBX Import: start importing /home/austin/Personal/NUBots/jo_pr_1/NUpbr/resources/robot/NUgus_esh.fbx
FBX version: 7400
	FBX import: Prepare...
		Done (0.000177 sec)

	FBX import: Templates...
		Done (0.000072 sec)

	FBX import: Nodes...
		Done (0.000054 sec)

	FBX import: Connections...
		Done (0.000028 sec)

	FBX import: Meshes...
		Done (0.097717 sec)

	FBX import: Materials & Textures...
		Done (0.000051 sec)

	FBX import: Cameras & Lamps...
		Done (0.000097 sec)

	FBX import: Objects & Armatures...
		Done (0.001406 sec)

	FBX import: ShapeKeys...
		Done (0.000018 sec)

	FBX import: Animations...
		Done (0.000018 sec)

	FBX import: Assign materials...
		Done (0.000060 sec)

	FBX import: Assign textures...
		Done (0.000009 sec)

	FBX import: Cycles z-offset workaround...
		Done (0.000006 sec)

	Done (0.110466 sec)

Import finished.
FBX Import: start importing /home/austin/Personal/NUBots/jo_pr_1/NUpbr/resources/robot/NUgus_esh.fbx
FBX version: 7400
	FBX import: Prepare...
		Done (0.000764 sec)

	FBX import: Templates...
		Done (0.000185 sec)

	FBX import: Nodes...
		Done (0.000024 sec)

	FBX import: Connections...
		Done (0.000029 sec)

	FBX import: Meshes...
		Done (0.097274 sec)

	FBX import: Materials & Textures...
		Done (0.000045 sec)

	FBX import: Cameras & Lamps...
		Done (0.000102 sec)

	FBX import: Objects & Armatures...
		Done (0.001600 sec)

	FBX import: ShapeKeys...
		Done (0.000018 sec)

	FBX import: Animations...
		Done (0.000017 sec)

	FBX import: Assign materials...
		Done (0.000055 sec)

	FBX import: Assign textures...
		Done (0.000009 sec)

	FBX import: Cycles z-offset workaround...
		Done (0.000005 sec)

	Done (0.111512 sec)

Import finished.
FBX Import: start importing /home/austin/Personal/NUBots/jo_pr_1/NUpbr/resources/robot/NUgus_esh.fbx
FBX version: 7400
	FBX import: Prepare...
		Done (0.000680 sec)

	FBX import: Templates...
		Done (0.000152 sec)

	FBX import: Nodes...
		Done (0.000164 sec)

	FBX import: Connections...
		Done (0.000152 sec)

	FBX import: Meshes...
		Done (0.096515 sec)

	FBX import: Materials & Textures...
		Done (0.000039 sec)

	FBX import: Cameras & Lamps...
		Done (0.000106 sec)

	FBX import: Objects & Armatures...
		Done (0.001808 sec)

	FBX import: ShapeKeys...
		Done (0.000018 sec)

	FBX import: Animations...
		Done (0.000017 sec)

	FBX import: Assign materials...
		Done (0.000053 sec)

	FBX import: Assign textures...
		Done (0.000010 sec)

	FBX import: Cycles z-offset workaround...
		Done (0.000005 sec)

	Done (0.110962 sec)

Import finished.
Traceback (most recent call last):
  File "/home/austin/Personal/NUBots/jo_pr_1/NUpbr/pbr/pbr.py", line 455, in <module>
    main()
  File "/home/austin/Personal/NUBots/jo_pr_1/NUpbr/pbr/pbr.py", line 73, in main
    misc_robots = [
  File "/home/austin/Personal/NUBots/jo_pr_1/NUpbr/pbr/pbr.py", line 74, in <listcomp>
    MiscRobot(
  File "/home/austin/Personal/NUBots/jo_pr_1/NUpbr/pbr/scene/misc_robot.py", line 28, in __init__
    self.construct(robot_info)   
  File "/home/austin/Personal/NUBots/jo_pr_1/NUpbr/pbr/scene/misc_robot.py", line 41, in construct
    with open(self.robot["kinematics_path"], "r") as file:
FileNotFoundError: [Errno 2] No such file or directory: '/home/austin/Personal/NUBots/jo_pr_1/NUpbr/resources/robot/GankenKun.json'
Saved session recovery to '/tmp/quit.blend'
Writing userprefs: '/home/austin/.config/blender/2.93/config/userpref.blend' ok

Blender quit

Here is the actual part chucking the wobbly I reckon

FileNotFoundError: [Errno 2] No such file or directory: '/home/austin/Personal/NUBots/jo_pr_1/NUpbr/resources/robot/GankenKun.json'

Attempting to run Blender w/ GUI resulted in a mostly empty scene, see image below. All of the objects in the scene are in the outliner.

Sylforen · 2026-02-21T12:23:53Z

Issue 2 SOLVED: Missing files in resources/robots - worked like a charm afterwards. Really solid detection for Robots + Balls in viewport. Detected multiple goal_posts at times, but nothing a little k-nn/IoU post-processing can't fix downstream.

ysims

I'd suggest removing the image curator tool, this was a bit of a hack to get a good dataset to train right before RoboCup and the bounding boxes weren't really working properly some of the time.

A few things

Are we still getting huge robot bounding boxes occassionally when they are very slightly in frame?
The goal posts should be fixed here, if there is an issue with them it needs to be part of this PR. The output of this tool should be a dataset that can be directly used to train an artificial neural network.
The NUbook page needs updating with this PR, particularly letting people know how to use the bounding box set up.
The intersection method is not really a great solution. It was the quickest thing we could do in the time we had before RoboCup. The main thing I don't like about it is that at the moment, you can't get both correct bounding box and segmentation outputs for the one raw NUpbr image, and each HDR needs the coloured blobs on the intersections. It would be better to use field dimensions to get them.

ysims · 2026-02-21T22:18:33Z

pbr/pbr.py

+        # Goal annotations  
+        # goal_annotations = [util.write_annotations(goal.obj, 1) for goal in goals]
+        # annotations += [ann for ann in goal_annotations if ann is not None]


Don't leave in commented out code

ysims · 2026-02-21T22:22:33Z

pbr/util.py

+    width = max_x - min_x
+    height = max_y - min_y
+
+    # Normalize coordinates


Suggested change

# Normalize coordinates

# Normalise coordinates

ysims · 2026-02-21T22:23:05Z

scripts/image_curator.py

+Image Curator Tool for NUpbr Dataset
+
+This tool allows you to review generated images with their annotations,
+visualize bounding boxes, and accept/reject images for your final dataset.


Suggested change

visualize bounding boxes, and accept/reject images for your final dataset.

visualise bounding boxes, and accept/reject images for your final dataset.

ysims · 2026-02-21T22:24:47Z

README.md

 </details>
+
+
+## BOUNDING BOX STUFF


better heading

ysims · 2026-02-21T22:25:44Z

README.md

+The current bounding box implementation in this branch does not integrate well with the segmentation side. The HDR must have blobs on the intersections in the following colours:
+
+- L: magenta [255, 0, 255]
+- T: cyan [0, 255, 255]
+- X: darker orange [255, 100, 0]
+
+The goal posts must be solid yellow [255, 255, 0] with just the posts and not the top bar or any other part of the goals.


It would be better to get the positions given we know field dimensions.

BaileyJoe · 2026-02-25T02:54:42Z

@ysims

Are we still getting huge robot bounding boxes occassionally when they are very slightly in frame?

As far as I can tell that isn't happening any more.

The goal posts should be fixed here, if there is an issue with them it needs to be part of this PR. The output of this tool should be a dataset that can be directly used to train an artificial neural network.

The only behaviour with goal posts that I've seen on the current version that might not be ideal is something like this where the bounding box is broken up because of it's partially blocked:

ysims and others added 10 commits June 30, 2025 13:34

ball radius

f164f54

bbox config

d5019da

vis tool

7f6299f

create and save annotations

e900e82

balls working?

2636a83

ball and robots good

e01fdef

YES

2582a4e

some info

3969858

.

df0a70a

mostly equisolid projection

284cd7e

BaileyJoe added the enhancement label Jan 15, 2026

added check for occluded balls

dfc1015

Sylforen self-assigned this Feb 20, 2026

Sylforen marked this pull request as ready for review February 21, 2026 12:21

Sylforen requested review from Sylforen, jpptm and ysims and removed request for Sylforen and jpptm February 21, 2026 12:24

ysims reviewed Feb 21, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add bounding box annotations#30

Add bounding box annotations#30
BaileyJoe wants to merge 11 commits intomainfrom
bailey/bboxes

BaileyJoe commented Jan 15, 2026

Uh oh!

Sylforen commented Feb 20, 2026

Uh oh!

Sylforen commented Feb 21, 2026

Uh oh!

ysims left a comment

Uh oh!

ysims Feb 21, 2026

Uh oh!

ysims Feb 21, 2026

Uh oh!

ysims Feb 21, 2026

Uh oh!

ysims Feb 21, 2026

Uh oh!

ysims Feb 21, 2026

Uh oh!

BaileyJoe commented Feb 25, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	visualize bounding boxes, and accept/reject images for your final dataset.
	visualise bounding boxes, and accept/reject images for your final dataset.

Conversation

BaileyJoe commented Jan 15, 2026

Adds bounding box annotations to output in YOLO format.

Uh oh!

Sylforen commented Feb 20, 2026

Uh oh!

Sylforen commented Feb 21, 2026

Uh oh!

ysims left a comment

Choose a reason for hiding this comment

Uh oh!

ysims Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

ysims Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

ysims Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

ysims Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

ysims Feb 21, 2026

Choose a reason for hiding this comment

Uh oh!

BaileyJoe commented Feb 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

BaileyJoe commented Feb 25, 2026 •

edited

Loading