-
Notifications
You must be signed in to change notification settings - Fork 1
Updates
Started to install HARK with ROS using instructions here: HARK-ROS
Also installed Kinect drivers from HARK here: HARK-KINECT
Following tutorials here: HARK-ROS Tutorial
Flowdesigner documentation: Flowdesigner
So we had this problem of getting flow designer to work because it updated in between us meeting. We tried the replacement program, but it was buggy and didn't meet our needs, so we had to revert back to the previous version of flow designer with the current version of the packages we need to run.
We completed the first tutorial of using flowdesigner, hark, and ROS. We published on a sample topic and were able to read the messages through ros.
Our problem now is with the fourth tutorial which actually does processing on a two channel wav file. The default response to an improper flow designer problem is a segfault which generally tells us nothing about what we did wrong in the process and is very frustrating and counterproductive to learning. Hopefully we find something soon to figure that out, but we're making progress considering the lack of appropriate resources we have.
We also read a paper and found a video on recognizing and localizing multiple speakers with an 8 microphone array. This seems replicable in our projects because each of the kinetics has 4 microphones.
Hopefully Hark's learning curve will level out pretty soon....
Downloaded HARK_recog
Using [HARK 1.2 documentation] (http://winnie.kuis.kyoto-u.ac.jp/HARK/hark-document-en-ver1.2.pdf)
Opened HARK_recog/recog.n and started to understand each node's purpose and how to use it. Attempting to use the kinect with the prebuilt network file, recog.n, for displaying sound localization only. Progress on using the kinect with HARK is ongoing.
Next Goal: After successfully connecting and using the kinect with HARK, the plan is to hook up HARK with ROS and get localization information publishing.
We wrote a network file, SoundLocMain.n, that takes input from a microphone and displays localization. The parameters for each node in the file were chosen to work with the kinect. The network file does display localization but the output is weird. We suspect it has to deal with the parameters that are set in the network file.
Inside this network file is also a ROS Publisher node, This node publishes the topic /HarkSource, whose message definition is found in hark_ros_stacks/hark_common_msgs/hark_msgs. This topic provides the x, y, z, theta, and power of a sound source. We are currently in the process of writing and building a ros node that subscribes to /HarkSource and echoes its messages.
So....
I emailed one of the people from the hark labs, and though I understood a little more about the kinect, I couldn't get soundlocalization to work. So I decided to download the turtlebot stack which had a localization network file for the kinect. I compared the parameters of the nodes there to the nodes we had on ours, and then tested it by saying "Hello" to the left and right of it. Luckily, it worked within about 30*.
That took wayyyyyyyyyyyyyy longer than it should, but we're finally getting somewhere; I'm pretty happy about that.
It's still not publishing to ROS for some reason. I tried reinstalling the ros stack and ros packages for flowdesigner and hark, but I continuously get the same error message...hark has no idea where my
We need to implement these networks:
JULIUS NETWORK - so we can recognize speech SOUND SEPARATION NETWORK - so that we can localize around 2 people speaking at the same time RECORDING NETWORK - so we can record and analyze sounds that come SOUND FILTER FOR A SET LENGTH - We think having a setup to filter out noise might benefit us in the long run. NETWORK TO LOCALIZE A PARTICULAR SOUND - Currently, we have no idea how to do this, but it might fare well if we can recognize a particular sound (beeps for the elevators would be SO nice).
Switched over to working on the turtlebot2. Currently working our node to rotate in the direction of the largest sound source. The following commands are used to start up the turtlebot and our node.
into bashrc:
export ROS_PACKAGE_PATH=/home/oliver/ros/hark_ros_stacks/hark-ros-turtlebot-stack- 1.2.0:/opt/ros/groovy/share:/opt/ros/groovy/stacks:/home/oliver/Code/BWI/SoundLocalization/rosbuild_ws
source /opt/ros/groovy/setup.bash
Assuming the following: Our node is built and the package path is exported in $ROS_PACKAGE_PATH Turtlebot is installed with ros-groovy hark-turtlebot source code is downloaded (no support for groovy and up)
roslaunch turtlebot_bringup minimal.launch rosrun hark_turtlebot_follower hark_turtlebot_follower rosrun hark_turtlebot_fdnetworks localization_ROS.sh rosrun sound_localization HARK_Source_echo
--Lost the laptop for my charger...so we're a little postponed for working this week. -Oliver Until then, we're reading up on ways of localizing sound for a single non-vocal sound source (which hark doesn't support) and integrating whatever code we write into HARK. For sound source localization, we've considered taking multiple samples of the sound we're trying to localize (such as an elevator beep), creating a corresponding structure with the value of the wave and the standard deviation from our samples at each point using a sliding window, or using some variation of fast Fourier transforms on a spectrum of our data.(We've ascertained this from looking over nodes in hark.) We've also considering implementing a basic neural network using a python package for computation.
##Week 12.1.2013 We're stuck in a limbo between balancing schoolwork and working on the project. We're stuck. We did a bit of re-organization on the main branch so that the package can be cloned to some catkin workspace and run appropriately as long as the appropriate packages are installed. We have had a HUGE problem setting up a network with sound separation, localization, and speech recognition working simultaneously.
So....we've run into a bunch of problems. The past two weeks