Skip to content

Just words of encouragement: the quality of diarization is very high #22

Description

@Manamama

An end-user mostly, but see e.g. numpy/numpy#27698 (comment), serengil/deepface#1614 that sometimes I delve into code.

  • I have just advised all and sundry to use yours: https://github.com/OpenMOSS/MOSS-Transcribe-Diarize.git
  • It installs (and runs) on Termux! (after one hour of upgrading av and transformers and the like: expected : Successfully installed audioop-lts-0.2.2 audioread-3.1.0 av-18.0.0 decorator-5.3.1 fastapi-0.139.2 joblib-1.5.3 librosa-0.11.0 moss-transcribe-diarize-0.1.0 pooch-1.9.0 python-multipart-0.0.32 scikit-learn-1.9.0 soundfile-0.14.0 soxr-1.1.0 standard-aifc-3.13.0 standard-chunk-3.13.0 standard-sunau-3.13.0 starlette-1.3.1 threadpoolctl-3.6.0 tokenizers-0.22.2 transformers-5.14.1 uvicorn-0.51.0 , on a humble Linux localhost 4.14.186+ #1 SMP PREEMPT Thu Mar 17 16:28:22 CST 2022 aarch64 Android "
~/Downloads/GitHub $ mtd-subtitle-web --model OpenMOSS-Team/MOSS-Transcribe-Diarize 
WARNING: linker: Warning: "/data/data/com.termux/files/usr/lib/python3.14/site-packages/optree.libs/libc++_shared-d523468d.so" unused DT entry: unknown processor-specific (type 0x70000001 arg 0x0) (ignoring)
INFO:     Started server process [9017]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://127.0.0.1:7860 (Press CTRL+C to quit)
INFO:     127.0.0.1:53816 - "GET / HTTP/1.1" 200 OK
INFO:     127.0.0.1:53816 - "GET /api/runtime HTTP/1.1" 200 OK
INFO:     127.0.0.1:53818 - "GET /api/jobs HTTP/1.1" 200 OK
INFO:     127.0.0.1:53818 - "GET /favicon.svg HTTP/1.1" 200 OK
INFO:     127.0.0.1:53838 - "POST /api/jobs HTTP/1.1" 200 OK
INFO:     127.0.0.1:53838 - "GET /api/jobs HTTP/1.1" 200 OK
INFO:     127.0.0.1:53838 - "GET /api/jobs/6ba85a15c0a7 HTTP/1.1" 200 OK
INFO:     127.0.0.1:53838 - "GET /api/jobs HTTP/1.1" 200 OK
Loading weights:   9%|██████████████▍                                                                                                                                           | 64/683 [00:00<00:02, 250.07it/s]INFO:     127.0.0.1:53838 - "GET /api/jobs HTTP/1.1" 200 OK
Loading weights:  82%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████                           | 563/683 [00:01<00:00, 416.18it/s]INFO:     127.0.0.1:53838 - "GET /api/jobs HTTP/1.1" 200 OK
Loading weights: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 683/683 [00:01<00:00, 352.47it/s]
INFO:     127.0.0.1:53838 - "GET /api/jobs HTTP/1.1" 200 OK
INFO:     127.0.0.1:53838 - "GET /api/jobs HTTP/1.1" 200 OK
[transformers] `MossTranscribeDiarizeProcessor` defines `feature_extractor_class = 'AutoFeatureExtractor'`, which is deprecated. Register the correct mapping in `AutoFeatureExtractor` instead.
INFO:     127.0.0.1:53838 - "GET /api/jobs HTTP/1.1" 200 OK
INFO:     127.0.0.1:53838 - "GET /api/jobs HTTP/1.1" 200 OK
INFO:     127.0.0.1:53838 - "GET /api/jobs HTTP/1.1" 200 OK
INFO:     127.0.0.1:53838 - "GET /api/jobs HTTP/1.1" 200 OK

Image
  • The GUI (after the transcription be done) is excellent too: I coded something with whisperx and PANN and plotly dynamic graphs years ago, while yours is clean and nifty.
  • The GUI helps one learn Chinese ;).
  • The diarization quality is excellent. Many files that had at best mangled most speakers to one, or the opposite: multiplied them as rabbits, via whisperx like pipelines even, now properly separate these to the right buckets. (Yes, I must read your paper how so, in detail, one day...)
1
00:00:05,330 --> 00:00:07,330
S01: Sognia's head, head!

2
00:00:08,060 --> 00:00:08,830
S01: Roder!

3
00:00:09,100 --> 00:00:11,100
S01: Sognia's head!

4
00:00:12,930 --> 00:00:13,660
S01: Roder!

5
00:00:13,700 --> 00:00:15,700
S01: Sognia's head, Sognia!

6
00:01:14,030 --> 00:01:18,830
S02: The man with the green jacket, the one who killed Zaper, find out what he knows.

7
00:01:38,760 --> 00:01:39,430
S03: Hi!

8
00:01:41,300 --> 00:01:42,830
S03: Hello, hello, A-Boy.

9
00:01:43,760 --> 00:01:44,530
S03: You alone?

10
00:01:46,000 --> 00:01:47,330
S04: Jätkaan blade raan.

11
00:01:48,230 --> 00:01:49,030
S04: Mitä se on?

12
00:01:50,260 --> 00:01:51,330
S04: Mennä, se on pitu vaan.

13
00:01:51,330 --> 00:01:51,830
S03: It's okay.

14
00:01:52,530 --> 00:01:53,130
S03: It's fine.

15
00:02:00,130 --> 00:02:01,530
S03: Wanna buy a lady a cigarette?

16
00:02:08,830 --> 00:02:10,260
S03: Oh, you don't even smile.

17
00:02:13,260 --> 00:02:14,630
S05: Didn't you hear your friends?

18
00:02:15,630 --> 00:02:16,730
S05: Don't you know what I am?

19
00:02:17,730 --> 00:02:18,430
S03: Yeah.

20
00:02:19,560 --> 00:02:20,930
S03: Guy eating rice.

21
00:02:22,760 --> 00:02:23,430
S03: What's that?

22
00:02:25,430 --> 00:02:26,130
S05: It's a tree.

23
00:02:27,230 --> 00:02:27,730
S03: Oh.

24
00:02:28,260 --> 00:02:29,730
S03: Never seen a tree before.

25
00:02:31,930 --> 00:02:32,630
S03: It's pretty.

26
00:02:35,430 --> 00:02:36,230
S05: It's dead.

27
00:02:38,630 --> 00:02:40,130
S03: Now, who keeps...

28
00:02:40,930 --> 00:02:42,430
S03: A dead tree?

29
00:02:50,360 --> 00:02:51,830
S03: You're not gonna kill me, are you?

30
00:02:52,160 --> 00:02:52,930
S05: Depends.

31
00:02:53,930 --> 00:02:54,930
S05: What's your model number?

32
00:02:55,760 --> 00:02:57,730
S03: Why don't you look into my eye and find out?

33
00:03:00,860 --> 00:03:02,130
S03: Oh...

34
00:03:02,930 --> 00:03:04,330
S03: You don't like girl girls.

35
00:03:08,230 --> 00:03:08,730
S03: Well...

36
00:03:09,430 --> 00:03:10,260
S03: I'm always here.

from a famous movie, doable without any optimization (scaling down, lower sized models, switches) on:
Image

  • 再次恭喜 !

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions