Archive for the 'pjmedia' Category

WebRTC Acoustic Echo Cancellation on PJSIP

image

Echo, unless you’re in a karaoke pub, is very annoying.

Over the years we have added support for various echo suppression and cancellation algorithm and libraries, like Speex and bdIMAD.

WebRTC, besides being a communications protocol on its own right, contains a powerful media component which in turn contains, among other things, acoustic echo cancellation algorithm implementation.

Try pjsip using WebRTC AEC on the development code by following the usage notes.

Eventually this will form part of 2.5 release (before you ask, no, we don’t have a definite timeframe for it).

Originally posted on http://blog.pjsip.org

pjsip adds support for OpenCORE AMR-WB AMR-NB codec

Support for more codecs is an ongoing request for pjsip.

Here’s a couple of of new ones from the AMR family: AMR-NB and AMR-WB.

It’s not part of any release yet, you have to get the latest code from the repository.

More details on the following tickets:

https://trac.pjsip.org/repos/ticket/1388

https://trac.pjsip.org/repos/ticket/1608

Let us know how you get along with it.

Port Your PJSIP Engine to BlackBerry 10 in Less Than 10 minutes

[This is a guest post from Gurtej Sandhu, developer relations at RIM]

If you haven’t already heard, the BlackBerry 10 countdown is on. If you have an existing application using PJSIP libraries, this is your opportunity to port your pjsip open source stack to BlackBerry 10 in a matter of minutes. As you may have already heard, Bob Cripps has successfully ported PJSIP to BlackBerry 10. Just very recently Bob has helped simplify building PJSIP for BlackBerry 10 by creating a set of executable scripts. This work has now all been committed to our BlackBerry github repository.

I took this opportunity to dig deep into building PJSIP for BlackBerry 10. As soon as I had my Linux environment up and running with all the prerequisites installed, I am happy to say that it took me less than ten minutes to build and load PJSIP BlackBerry 10 Cascades sample project to my BlackBerry 10 Dev Alpha device. So please don’t try to reinvent the wheel – dive right into this github repository to port PJSIP to BlackBerry 10. Remember to follow the README instructions as they are very important. You can also follow the instructions in PJSIP porting guide knowledge base article.

If you run into any issues in porting PJSIP to BlackBerry 10 you can send me a tweet @_GurtejSandhu or write your comment below and I will be happy to assist.

Again, huge kudos to Bob Cripps for contributing his recent work in simplifying building PJSIP for BlackBerry 10.

Success stories:

http://devblog.blackberry.com/2012/07/voip-development-on-blackberry-10/

BlackBerry 10 Development:

https://developer.blackberry.com/develop/platform_choice/ndk.html

http://devblog.blackberry.com/2013/01/blackberry-10-voip/

Related Posts

Call for Symbian S60 testers for pjmedia APS direct

As we finish the current iteration of Symbian S60 implementation, most
importantly the implementation of APS-direct, we need all of you to
participate in testing it. This is from experience we know mobile
devices are very tricky and can behave differently from one firmware
to another.

So we need as many Symbian S60 3rd Edition phone as possible. The test will involve installing a test application, checking a few things, and filling in a report form

To do this we need your IMEI and a few other details, this is because
the requirements of Symbian Signed, we need to ‘burn’ your IMEI into
our test application.

Don’t reply with your details to the public list, instead fill in this
form: http://l.teluu.com/symbiantesting

We can’t promise everyone will be accepted, because there is a hard
limit to the number of IMEIs we can burn, and we’ll prioritize
variation of devices, rather than first come first serve.

If you can get your friends family neighbours to join in that will be
even better.

If you any questions, just leave a comment and.

Thank you all!

Analyzing the PJMEDIA Tone Generator Algorithms Performance

We have just finished some refactoring in the PJMEDIA tone generator, due to some problems identified by ticket 619. The main problem was the bad quality tone generated by the fixed-point sine generator algorithm, causing some failed detections by the remote end. While fixing this, we also addressed other problems such as noticeable thump noise at the beginning and the end of a tone, and difference in audio level produced by the fixed point compared to the floating point generator.

The results are these enhancements to the tone generator, implemented by ticket 619:

  • introduced a new fixed-point sine generator backend, based on the CORDIC algorithm. While this algorithm is generally slower than the existing fixed-point backend (although it is still faster than the floating-point algorithm on platforms that lack FP support), it does produce more precise output, and more importantly, the precision is tuneable so that application can balance between precision and speed.
  • made the backend algorithm selectable via compile time setting.
  • added fade-in and fade-out to the tone generation to eliminate/reduce the thump noise.

To verify these, we analyzed the performance (in terms of both speed and precision) of the various backends. To see the result, please see the Analyzing the PJMEDIA Tone Generator Algorithms Performance in our wiki page.

New low-complexity echo suppressor for mobile devices

As many have probably experienced it first hand, the echo suppressor (ES) in pjmedia sucks, to put it mildly. And this has made matters worse since ES plays such a major role in PDA’s and/or mobile devices/smartphones, since echoes are heavily present on these devices and we can’t really put the Accoustic Echo Canceller (AEC) on these devices due to the high processing requirements of the AEC (see our results in Evaluating PJMEDIA Performance article).

So we scrapped the old echo suppressor and wrote another one from scratch. And I think this one works quite well.

First of all, it’s low-complexity so it doesn’t require high processing requirements. We tested on a PocketPC 2003 PDA with 312MHz StrongARM CPU, it only uses less than 0.5% (half percent) of CPU. So it’s very affordable for these devices.

And most importantly, I think it works! It suppresses echo significantly, it lets non-echo audio pass through, and also it allows double-talk where both parties are talking at the same time. This is a very important to allow enjoyable conversation between two people, so we’ve made sure that the new echo suppressor supports this.

For those who care with the details, the ES works by comparing the audio level pattern in the input signal from the microphone with the audio level pattern in the playback signal during the learning process. It calculates the correlation values for each tail position up to the configured echo tail length, and finds out which tail position has the best correlation value. This position is then marked as the echo position. While doing this, the ES also notes the gain factors, that is how much de-amplification to be applied to the input signal in order to remove the echo from the input. Once the tail position is found, the ES then applies the appropriate gain factor according to the state of the conversation (e.g. I’m talking you’re silent, you’re silent I’m talking, we’re both talking, etc.).

This works pretty well. On the downside though, we noticed that the ES still seems to let a quiet echo to pass and transmitted back to remote party, depending on the signal level at the time. For now we allow this to happen since the echo level is very low and  doesn’t seem to be too annoying, and lacking enough experience with the new algorithm we don’t want to make it cut signals too aggressively yet since it may inadvertently cut the “good” signal.

The new echo suppressor is available in SVN as pjmedia/echo_suppress.c, and it will be included in the next release.

Evaluating PJMEDIA Performance

Performance is one of the most common questions that developers asked. We’ve been asked questions like, can I run X on platform Y, or how much MIPS required to run component X, and so far our answer would be I don’t know, or at best, why don’t you try it yourself and see what happens.

So we decided to do a bit of benchmarking for pjmedia for several platforms that we have, and you can see the result here:

http://trac.pjsip.org/repos/wiki/PJMEDIA-MIPS.

The test covers various platforms, and sometimes the same H/W platform but different OS’es (Linux/gcc vs Windows/Visual Studio) to see how they fare, and of course various PJMEDIA components.

As the article says, there are some drawbacks about the test method used, but nevertheless I think it could be useful to see the rough CPU requirements of various PJMEDIA components. Certainly it has been useful to us, and in fact there have been some surprises with the results. For example, we expected that WSOLA (Waveform Similarity Overlap and Add, the algorithm that we use to conceal packet lost and to handle clock drifts) to take large chunks of CPU usage, but turns out it is quite fast. Also resampling with small filter looks to be quite affordable too.

We’ll look forward to measure the performance on more platforms (notably, Symbian), and we’ll keep track of the performance for future releases. In the meantime, enjoy the article.

PJSIP 0.9 is Released: Audio Latency, TURN implementation, IPv6, G.722, and More

Finally, after months of delay, PJSIP version 0.9.0 is released. This has been the longest gap (8 months) between releases, and consequently it has the most modifications in it (there have been 100+ tickets done on this release).

Some of the new features in this release:

  • many improvements in the audio, to reduce audio latency, to have better compatibility with more target platforms (Windows Vista issues have been fixed, as well as sporadic error reports for ALSA), and to maintain the audio quality against impairments such as clock drifts, bursty sound device, and of course, packet loss. Compared to version 0.8, I think we’ve improved audio latency by few hundred milliseconds.
  • support for TURN-07 in PJNATH, either as standalone client/server library, standalone client/server application (for testing purposes), or integrated with ICE-19. Just as we were the first to release open source ICE library, I think this is also the first open source TURN implementation out there. Unfortunately we haven’t had time to update it to TURN-08 as this draft was released late during our QA phase, but we’ll update it as soon as possible.
  • fixed the ICE offer/answer rules.
  • support for IPv6.
  • support for Secure RTP (SRTP)
  • better support for Windows Mobile target. We have new and more usable sample application (PocketPJ) and GSM and Speex codec should now be available for this target.
  • better support for Symbian S60 target. There is a more thorough Symbian tutorial available, and GSM and Speex codec should now be available for this target too.
  • implementation of G.722 codec.
  • support for RTCP Extended Report (XR)
  • and many more.

For more information, start from PJSIP download page. Get it while it’s hot!

Doing it in Stereo

While PJMEDIA have supported stereo audio since day one, it had come with few limitations, for example this capability was not exported in PJSUA-LIB API, and once stereo mode is set, everything must be set to stereo too. These have been fixed in the latest SVN now. There is a new configuration field in pjsua_media_config to set the number of channels configuration for both the sound device and the conference bridge (the default is one of course), and more importantly, the conference bridge can now allow media ports with different channel count setting to be registered to it.

As an example of the new capability, application can instantiate both the sound device and the bridge in stereo. If the call is established with a stereo codec (for stereo we only have L16 codec for now), then transmission will be stereo, and if the call is established with the usual mono codec (G.711, G.722, Speex, etc. And oh yes, we do have G.722 now!), then the bridge will correctly perform the stereo – mono conversion, mixing the audio channels as necessary.

Having said all that, you may ask, why? Why bother with stereo at all?

You’re right, stereo is uncommon in VoIP. But nowadays SIP is used in broader industries than just VoIP. For example, the European Broadcasting Union (EBU) is currently looking to standardize Audio Contribution over IP (N/ACIP) which is based on standard based protocols such as SIP and RTP, and stereo encoding comes as one of the mandatory requirements. Also on different industry, we’re currently looking to integrate PJSIP as a VoIP platform in Virtual World games, in the open source VoIP for Virtual World project. This project is still in its very early stage, as it’s been published literally just couple of weeks ago. Here I guess stereo will play quite an important role, because of the positional audio requirement. And still there are more use cases for stereo, for example in SIP to radio gateways where people do want to transmit different audio in the left and right channel. The last use case probably is niche, but people do ask for that.

So in conclusion, stereo does have a use case. And I’m glad that we have a pretty smooth support for that.

And how can we enable stereo in the application? Well just set pjsua_media_config.channel_count to 2, and that’s just about it! More advanced scenarios are possible of course, but at least it’s very easy to get started with it.

Securing VoIP: SRTP Support in PJSIP

PJSIP now has SRTP support in SVN trunk (hurray!). For more information about compatibility, how to use, and what have been done, please see the SRTP wiki in
http://trac.pjsip.org/repos/wiki/SRTP

We’ve tested it against couple of phones that support SRTP and it looks good (apart from SRTCP, which one phone doesn’t support and the other we don’t think it implements SRTCP correctly). If you have phones which support SRTP, it would be great if you could give it a try, I’ll be thrilled to hear your result.

Enjoy it while it’s hot!