OpenCV for Processing

[kasperkamperman]

OpenCV for Processing uses the direct OpenCV Java bindings instead of JavaCV. Please only include either one library, because Processing gets confused if two OpenCV packages are imported.

Is there a way to rewrite the examples code using OpenCV for Processing, to use the OpenCV included in DeepVision.
I like students to play around with this, but installing/removing libraries might be too much hassle.

How would I import OpenCV with this library.
import gab.opencv.*;
And what would be the OpenCV object instead of OpenCV cv; .

Or is it not that simple?

[cansik]

@kasperkamperman This is still an unsolved problem, but I guess it would be possible to release a deep-vision library without opencv included, but to use the one from the opencv-processing package. So it would be possible to have both packages installed. Currently there is no required packages option for libraries in Processing, that's why I do ship the opencv binaries for every package.

[kasperkamperman]

Is there a reason to separate the two libraries? I get it from a technical point of view (deep learning algorithms v.s normal), but for a designer/artist that is experimenting maybe it doesn't matter so much.

Maybe DeepVision could also include the regular examples of the OpenCV library.

I can imagine you don't want to touch the other one, because I believe it's used in a book as well.

[cansik]

Yes, you are right, it could also be a combined library. But the API to the designer/artist is not the same for the two libraries. So it would take some time and thinking on how to combine the two libraries. Maybe to create a new and modern computer vision library for Processing, combining both libraries. With that approach, we would not break any existing script and have more freedom to design the new library.

[cansik]

@kasperkamperman As a workaround: I have create a bare option for the build processing of opencv-processing (opencv_processing-bare.zip). This will not include the dependencies and you can install the library side by side with deep-vision-processing, but you have to manually install it. But now you have the possibility to include deep-vision-processing and opencv-processing at the same time.

⚠️ Please be aware that if you only import the gab.opencv.* namespace, the opencv-binaries from deep-vision are not loaded. You have to explicitly tell processing that you want to load these binaries by adding the following import (or any import to org.opencv.*):

import org.opencv.core.Mat;

Yes, this is a hack and not great, but at least now it's possible to combine both libraries, until we find a better solution or Processing fixes the dependency behaviour.

Here now a full example using opencv-processing & deep-vision:

import org.opencv.core.Mat;
import gab.opencv.*;

import ch.bildspur.vision.*;
import ch.bildspur.vision.result.*;

PImage image;

OpenCV opencv;

DeepVision deepVision = new DeepVision(this);
YOLONetwork yolo;
ResultList<ObjectDetectionResult> detections;

int textSize = 12;

void setup() {
  size(768, 576);
  colorMode(HSB, 360, 100, 100);

  image = loadImage("https://github.com/pjreddie/darknet/raw/master/data/dog.jpg");

  opencv = new OpenCV(this, image);
  opencv.brightness(128);

  yolo = deepVision.createYOLOv4Tiny();
  yolo.setConfidenceThreshold(0.3f);
  yolo.setup();

  detections = yolo.run(opencv.getSnapshot());
}

void draw() {
  background(0);

  image(opencv.getSnapshot(), 0, 0);

  strokeWeight(3f);
  textSize(textSize);

  for (ObjectDetectionResult detection : detections) {
    int hue = (int)(360.0 / yolo.getLabels().size() * detection.getClassId());

    noFill();
    stroke(hue, 80, 100);
    rect(detection.getX(), detection.getY(), detection.getWidth(), detection.getHeight());

    fill(hue, 80, 100);
    rect(detection.getX(), detection.getY() - (textSize + 3), textWidth(detection.getClassName()) + 4, textSize + 3);

    fill(0);
    textAlign(LEFT, TOP);
    text(detection.getClassName(), detection.getX() + 2, detection.getY() - textSize - 3);
  }
}