Archive for the 'VoIP' Category

Port Your PJSIP Engine to BlackBerry 10 in Less Than 10 minutes

[This is a guest post from Gurtej Sandhu, developer relations at RIM]

If you haven’t already heard, the BlackBerry 10 countdown is on. If you have an existing application using PJSIP libraries, this is your opportunity to port your pjsip open source stack to BlackBerry 10 in a matter of minutes. As you may have already heard, Bob Cripps has successfully ported PJSIP to BlackBerry 10. Just very recently Bob has helped simplify building PJSIP for BlackBerry 10 by creating a set of executable scripts. This work has now all been committed to our BlackBerry github repository.

I took this opportunity to dig deep into building PJSIP for BlackBerry 10. As soon as I had my Linux environment up and running with all the prerequisites installed, I am happy to say that it took me less than ten minutes to build and load PJSIP BlackBerry 10 Cascades sample project to my BlackBerry 10 Dev Alpha device. So please don’t try to reinvent the wheel – dive right into this github repository to port PJSIP to BlackBerry 10. Remember to follow the README instructions as they are very important. You can also follow the instructions in PJSIP porting guide knowledge base article.

If you run into any issues in porting PJSIP to BlackBerry 10 you can send me a tweet @_GurtejSandhu or write your comment below and I will be happy to assist.

Again, huge kudos to Bob Cripps for contributing his recent work in simplifying building PJSIP for BlackBerry 10.

Success stories:

BlackBerry 10 Development:

Related Posts

pjsip version 1.5 is released with TLS Rewrite, TLS for Symbian, QoS, and MWI Support

Version 1.5 has just been released with the following features.

SSL/TLS Rewrite

A new secure socket abstraction is implemented in PJLIB. The API is implemented using the native CSecureSocket for Symbian platform and OpenSSL for other platforms. With this API, new type of implementations (such as native Windows SSPI)  could be written in the future.

The SIP TLS transport has been rewritten to make use of this secure socket API, while maintaining the existing SIP TLS transport API. The secure socket API will also make way for other SSL/TLS based transports in the future, such as TLS TURN client connection.

QoS Framework

All transports in the library (such as SIP UDP/TCP/TLS transports, UDP media transport, and STUN/TURN/ICE transports) have been equipped with QoS (Quality of Service) settings. The QoS framework abstracts QoS technologies such as the Type of Service/DiffServ Code Point (ToS/DSCP) fields, Wi-Fi Multimedia (WMM) priorities, and IEEE 802.1p tagging (via SO_PRIORITY) in a generic manner, while providing flexibility for applications to adjust the settings manually if wanted.

The QoS framework has been tested on Symbian, Windows Mobile 6, Linux, and MacOS X. Note that currently it is not available on Windows XP and later.

Please see the new QoS wiki page for more info.

Message Summary/Message Waiting Indication (MWI) Support

Added support for both subscription based MWI (RFC 3842) and unsolicited MWI that is used by a popular PBX. For more information please see ticket #982.

Presence Enhancements

Ticket #937 among other things implemented automatic buddy’s presence resubscription upon receiving several specific termination causes. Ticket #411 and #364 improved the PUBLISH request handling.

SIP INVITE/CANCEL Destination Fixes

Ticket #917 and #936 fixed the following problems:

  • CANCEL request may be sent to different server than the INVITE when DNS SRV is used
  • INVITE request retry because of 401/407 response may be sent to different server than the INVITE when DNS SRV is used
  • CANCEL request will be sent with UDP if the INVITE was sent with TCP because of 1300 bytes message size/MTU limit (it must be sent with the same transport)

Please get the new version from PJSIP download page as usual.

Rich Internet Telephony Application, Anyone?

pjsip has always been cross-platform, basically it runs anywhere. Moreover,
we interpreted platform liberally, so in addition to multiple operating
systems and processor, we also have runtimes-as-a-platform (RaaP?) like
the Python VoIP API support (and the community has expanded along this line to Java SIP API and C# SIP SDK).

There is a new breed of platform coming to the desktop, broadly called Rich
Internet Application (RIA). Names like Microsoft Silverlight, Adobe AIR,
Sun JavaFX have been wading in this area. On my desktop I got Twhirl, a
Twitter client built on top of AIR.

So is this platform going to be relevant to SIP clients, I wonder? Would it
fit into a IP phone developer’s strategy? Is a RIA softphone possible or
even desirable? Would we call these things Rich Internet Telephony
Applications (RITA)?

Click your thoughts on the poll below and let me know what you think! (UPDATE: The poll application seems to be not working for some people – here is the direct link to the: Rich Internet Telephony Application Poll)

Presenting at CELF Embedded Linux Conference Europe 2008

In the world of embedded Linux, one of the players is a group called Consumer Electronics Linux Forum (CELF). CELF was formed five years ago by the big manufacturers: Matsushita, Sony, Hitachi, NEC, Philips, Samsung, Sharp, and Toshiba.

One of the events they run is Embedded Linux Conference (ELC) series, which has been running the past three years and have two version, one in the US and one in Europe.

For the CELF ELC Europe 2008 we’re going to present a session on pjsip as an Open Source Compact SIP and Media Stack.

What’s the connection of pjsip to embedded Linux? Because pjsip is compact and lightweight it runs well in embedded Linux. We’re also going to talk about a recent case where Linux was used as the operating system for an IP Phone, and pjsip as the SIP stack, with pjmedia as the media stack.

Anyway, if you are anywhere near Ede, Netherlands on 6-7 November 2008, look us up!

UPDATE: The presentation is now available: pjsip: Open Source Compact SIP and Media Stack.

Analyzing the PJMEDIA Tone Generator Algorithms Performance

We have just finished some refactoring in the PJMEDIA tone generator, due to some problems identified by ticket 619. The main problem was the bad quality tone generated by the fixed-point sine generator algorithm, causing some failed detections by the remote end. While fixing this, we also addressed other problems such as noticeable thump noise at the beginning and the end of a tone, and difference in audio level produced by the fixed point compared to the floating point generator.

The results are these enhancements to the tone generator, implemented by ticket 619:

  • introduced a new fixed-point sine generator backend, based on the CORDIC algorithm. While this algorithm is generally slower than the existing fixed-point backend (although it is still faster than the floating-point algorithm on platforms that lack FP support), it does produce more precise output, and more importantly, the precision is tuneable so that application can balance between precision and speed.
  • made the backend algorithm selectable via compile time setting.
  • added fade-in and fade-out to the tone generation to eliminate/reduce the thump noise.

To verify these, we analyzed the performance (in terms of both speed and precision) of the various backends. To see the result, please see the Analyzing the PJMEDIA Tone Generator Algorithms Performance in our wiki page.

Version 1.0-rc2 is released

“Good news, everyone”.

PJSIP version 1.0-rc2 is ready. There’s not too many updates here (which is good, by the way), only 9 tickets on this release, but nevertheless some important bugs were fixed. So we’re inching closer towards the 1.0 release.

There’s not much else to discuss here, please go straight to the download page for the details.

Version 1.0-rc1 is released with new Python SIP, Nokia APS support, and IPP codecs

“Good news, everyone” [Professor Hubert J. Farnsworth, Futurama]

This is an interim release, intended to mark the end of features development in the trunk. From now on, it will be tests and bug fixes only, until we reach 1.0. This will be our first proper stable release, and it will be given a separate branch, to isolate it from bleeding edge developments in the trunk.

Because of that, there has been a bit of “pressure” to stuff in as much features as possible on this release, since this is the last change to include them in 1.0. Here are some of them:

  • Integration of Intel® IPP Codecs.  This brings us with bunch of new codecs into PJMEDIA, such as G.722.1, G.723.1, G.726, G.728, G.729A, AMR NB, and AMR WB. Basically the lot! For more info about this integration, please see here.
  • New Python API. We discussed this on this blog a month ago here, basically it’s a new Python API for PJSUA-LIB, it’s much easier to use, and it also has a more thorough documentation/tutorial. Please check that out.
  • Nokia APS Support. The Nokia Audio Proxy Server is a wrapper to Nokia S60 sound device, it has much lower latency than Symbian MMF API (the traditional sound device that we support), and it also opens up support for device’s native codecs such as AMR, G.729, and iLBC which we can use in the future.  Although this API has been deprecated by Nokia in FP2, still there are lots of S60 and FP1 devices out there, so this is worth supporting.
  • New Echo Suppressor. Good for mobile devices, we discussed this in this blog here.

And some more. For more information regarding this release, please visit the download page.


New low-complexity echo suppressor for mobile devices

As many have probably experienced it first hand, the echo suppressor (ES) in pjmedia sucks, to put it mildly. And this has made matters worse since ES plays such a major role in PDA’s and/or mobile devices/smartphones, since echoes are heavily present on these devices and we can’t really put the Accoustic Echo Canceller (AEC) on these devices due to the high processing requirements of the AEC (see our results in Evaluating PJMEDIA Performance article).

So we scrapped the old echo suppressor and wrote another one from scratch. And I think this one works quite well.

First of all, it’s low-complexity so it doesn’t require high processing requirements. We tested on a PocketPC 2003 PDA with 312MHz StrongARM CPU, it only uses less than 0.5% (half percent) of CPU. So it’s very affordable for these devices.

And most importantly, I think it works! It suppresses echo significantly, it lets non-echo audio pass through, and also it allows double-talk where both parties are talking at the same time. This is a very important to allow enjoyable conversation between two people, so we’ve made sure that the new echo suppressor supports this.

For those who care with the details, the ES works by comparing the audio level pattern in the input signal from the microphone with the audio level pattern in the playback signal during the learning process. It calculates the correlation values for each tail position up to the configured echo tail length, and finds out which tail position has the best correlation value. This position is then marked as the echo position. While doing this, the ES also notes the gain factors, that is how much de-amplification to be applied to the input signal in order to remove the echo from the input. Once the tail position is found, the ES then applies the appropriate gain factor according to the state of the conversation (e.g. I’m talking you’re silent, you’re silent I’m talking, we’re both talking, etc.).

This works pretty well. On the downside though, we noticed that the ES still seems to let a quiet echo to pass and transmitted back to remote party, depending on the signal level at the time. For now we allow this to happen since the echo level is very low and  doesn’t seem to be too annoying, and lacking enough experience with the new algorithm we don’t want to make it cut signals too aggressively yet since it may inadvertently cut the “good” signal.

The new echo suppressor is available in SVN as pjmedia/echo_suppress.c, and it will be included in the next release.

Python SIP Take Two (Part 1)

Python is here again!

More than a year ago I wrote Python binding for PJSIP. It was alright, we can have some Python applications done using this wrapper. But, it’s not really having the impact that I expected. I’m talking about programming experience here and not popularity or things like that.

I mean, Python programming is supposed to be easy, and above all else, fun and enjoyable. But the wrapper is not doing that. I don’t know what it is, maybe it still smells too much like C, or maybe it’s the lack of documentation, or both, or something else. Bottom line I was not too impressed, so I’ve always thought about redoing it.

And now it’s done!

With the new module, we now do the absractions in two layers. The lower layer is the _pjsua C extension which exports PJSUA API to Python. This is similar to how the old Python extension was implemented. But now we also add a higher layer abstraction, object oriented, pure Python module on top of this. And that is the module.

The pjsua module provides high level API for constructing Session Initiation Protocol (SIP) multimedia user agent applications (a.k.a Voice over IP/VoIP softphones). It wraps together the signaling, media, and NAT traversal functionality into easy to use call control API, account management, buddy list management, presence, and instant messaging, along with multimedia features such as local conferencing, file streaming, local playback, and voice recording, and powerful NAT traversal techniques utilizing STUN, TURN, and ICE.

Hopefully it really is easier and more fun to use now.

Ready to go? We’ve also created more thorough documentation this time, start your development from this page: Python SIP Tutorial

And stay tuned for next parts of this post, we may have some interesting applications to publish (hints: I’m thinking about SIP client program for Nokia S60 platform, with PyS60. Wish me luck!).

Evaluating PJMEDIA Performance

Performance is one of the most common questions that developers asked. We’ve been asked questions like, can I run X on platform Y, or how much MIPS required to run component X, and so far our answer would be I don’t know, or at best, why don’t you try it yourself and see what happens.

So we decided to do a bit of benchmarking for pjmedia for several platforms that we have, and you can see the result here:

The test covers various platforms, and sometimes the same H/W platform but different OS’es (Linux/gcc vs Windows/Visual Studio) to see how they fare, and of course various PJMEDIA components.

As the article says, there are some drawbacks about the test method used, but nevertheless I think it could be useful to see the rough CPU requirements of various PJMEDIA components. Certainly it has been useful to us, and in fact there have been some surprises with the results. For example, we expected that WSOLA (Waveform Similarity Overlap and Add, the algorithm that we use to conceal packet lost and to handle clock drifts) to take large chunks of CPU usage, but turns out it is quite fast. Also resampling with small filter looks to be quite affordable too.

We’ll look forward to measure the performance on more platforms (notably, Symbian), and we’ll keep track of the performance for future releases. In the meantime, enjoy the article.