Feature: support webcams for behavior recording

[t-sasatani]

Updates

• mio/behavior_cam.py: Multiprocess behavior cam runner. Very similar to stream_daq.
• mio/io.py: Add cv2 backend to videowriter
• mio/cli/usbcam.py: CLI for behavior camera. Mostly matches the stream capture pattern, but has additional interfaces for camera listing and selection.
• mio/data/config/camera/*.yaml: Example config for camera
• mio/devices/mocks.py: Added USBCamMock. Similar to OKDevMock but uses .npz for data as this is mainly generated by the test fixture.
• mio/devices/usbcam.py: Helper functions for the camera.
• mio/models/usbcam.py: Model for the webcam.
• mio/utils.py, mio/stream_daq.py: Move exact_iter to utils.py for reusing.

Misc

This started from porting functionalities of the following repo:
https://github.com/MarcelMB/ELP_behavior_wide_lens_camera

---

📚 Documentation preview 📚: https://miniscope-io--144.org.readthedocs.build/en/144/

[coveralls]

Pull Request Test Coverage Report for Build 19775939086

Details

• 24 of 298 (8.05%) changed or added relevant lines in 8 files are covered.
• 1 unchanged line in 1 file lost coverage.
• Overall coverage decreased (-7.7%) to 71.388%

---

Changes Missing Coverage	Covered Lines	Changed/Added Lines	%
mio/cli/main.py	0	2	0.0%
mio/models/usbcam.py	0	15	0.0%
mio/io.py	13	29	44.83%
mio/cli/usbcam.py	0	49	0.0%
mio/devices/usbcam.py	0	53	0.0%
mio/behavior_cam.py	0	139	0.0%

Files with Coverage Reduction	New Missed Lines	%
mio/cli/main.py	1	0.0%

Totals
Change from base Build 19747854224:	-7.7%
Covered Lines:	1991
Relevant Lines:	2789

---

💛 - Coveralls

[sneakers-the-rat]

will take a look after holiday, but yes I want to refactor all the devices into devices like this.

[t-sasatani]

Thanks, that sounds great to me. The quality fixes on my end for getting this into main will probably happen after finishing the upcoming experiments, so no rush at all.

[t-sasatani]

Also, more as a note to myself, this is pretty much the same pattern as stream_daq.py, so I will want to find what can be designed to be reusable to minimize repetition.

[sneakers-the-rat]

I think u are gonna have fun with noob

[sneakers-the-rat]

I think ultimately the interface we will want is one where all the different types of cameras have a shared interface, and the config determines what class/etc. Is used to implement that interface.

So we would want to be able to do mio stream capture --device wireless-miniscope-1 as well as mio stream capture --device usb-webcam-1 rather than have a separate command for each type of device. As you say, there is more overlap than non-overlap in the code, maybe the only thing that's unique is the initial stage of pulling data. I think the exception might be the SD-card scopes, which I think there is only one of, because they're not streaming devices.

then stuff for specific device types would be like mio device usbcam --list and mio device usbcam create and whatnot.

BC eventually we will want to be able to do like mio stream capture --device cam1 --device cam2 and so on to do simultaneous sync'd capture, and when making a GUI, having a common programming interface will be very very useful.

[t-sasatani]

Makes sense. I think about the combining thing a bit later.

[t-sasatani]

Something that might be a rather fundamental limitation here is that the skvideo.io still drops a few frames for colored frames. Now I added an optional cv2 backend to the VideoWriter to get around this, but I guess that is not great.

#138

[sneakers-the-rat]

Ope I missed that. Yep that's a must fix

[sneakers-the-rat]

@t-sasatani says: ready except for testing

[t-sasatani]

Will be pretty noisy for a while as I'm having trouble reproducing the skvideo frame drop.

[t-sasatani]

Probably getting too obsessed, but one issue was that the moov atom isn't set properly on Windows/skvideo after ending recording with Ctrl-C. It seems like Ctrl-C directly kills ffmpeg on Windows, and the closure isn't handled properly.

But I can't think of a way to emulate this in CI/CD in a way that makes sense, so I just changed to use q to stop.
I also added a check for metadata and frame-count alignment, so at least mismatches are detectable.

[t-sasatani]

skvideo on Mac seems to handle this correctly, so the frame and metadata row count match. That's probably why this skvideo issue didn't happen when using stream_daq.

[t-sasatani]

So we could live without cv2 now, but I also don't feel skvideo is fully reliable, so maybe keeping multiple backends makes the most sense for now. At least for mio.