Archive for the 'smartphone' Category

pjsip for BlackBerry PlayBook: Native SIP client may be closer than you think

One of the most frequently ask questions we got is: does pjsip support BlackBerry? Are there plans to do so?

We always answered no, because the current BlackBerry devices only supports Java and even then there seem to be lack of multimedia access (essential for audio/video capture and playback) and direct socket access (for media streaming).

Recently however, with the BlackBerry Playbook using a different operating system, they have also announced that native C/C++ development kit will be available later this year.

So, maybe it will be not long now until we can answer ‘Yes’. Well, at least it can run on the BlackBerry PlayBook, if not all their smartphones.

Version 1.1 is released with support for Nokia native codecs and new audio device API

Good news, everyone!

Finally, after months of developments (read: delays!), version 1.1 is ready for your download. This release contains major feature enhancements, namely support for Nokia native codecs (we use code name APS-Direct for this feature) and a new Audio Device API.

APS-Direct is our codename for functionalities to use the hardware codecs that are supported by sound devices e.g. Nokia Audio Proxy Server (APS) and/or VoIP Audio Services (VAS) directly, bypassing media processing in PJMEDIA. The Nokia APS and VAS support codecs such as G.711 (PCMA and PCMU), G.729, iLBC, and AMR-NB, though the availability of these codecs may vary according to the handset types. There are significant benefits of using these codecs instead of software codecs (in PJMEDIA-CODEC), with the main benefits are performance (hardware vs software codecs, latency) and the given codec licenses/royalties. Due to these benefits, the ability to use these codecs in PJSIP applications is very desirable, hence the support.

This has been a major development in PJMEDIA, as traditionally PJMEDIA works with PCM (linear, L16) audio samples. With APS-Direct, audio frames from the sound device are in encoded format, so some components along the media path need to be updated to support encoded frame format. Understandably, some features cannot be used when encoded audio is active, for example the mixing feature of the conference bridge. Please see APS-Direct wiki for more information.

Half way during APS-Direct development, we discovered that the existing sound device abstraction API couldn’t cope with the new features, for example handling of encoded frames, setting the audio device routing, etc. We could of course patch it here and there, but we decided that creating a new one would be a much better alternative. So a new Audio Device API was developed.

For more information about this release, start from the PJSIP Download page. Enjoy!

New low-complexity echo suppressor for mobile devices

As many have probably experienced it first hand, the echo suppressor (ES) in pjmedia sucks, to put it mildly. And this has made matters worse since ES plays such a major role in PDA’s and/or mobile devices/smartphones, since echoes are heavily present on these devices and we can’t really put the Accoustic Echo Canceller (AEC) on these devices due to the high processing requirements of the AEC (see our results in Evaluating PJMEDIA Performance article).

So we scrapped the old echo suppressor and wrote another one from scratch. And I think this one works quite well.

First of all, it’s low-complexity so it doesn’t require high processing requirements. We tested on a PocketPC 2003 PDA with 312MHz StrongARM CPU, it only uses less than 0.5% (half percent) of CPU. So it’s very affordable for these devices.

And most importantly, I think it works! It suppresses echo significantly, it lets non-echo audio pass through, and also it allows double-talk where both parties are talking at the same time. This is a very important to allow enjoyable conversation between two people, so we’ve made sure that the new echo suppressor supports this.

For those who care with the details, the ES works by comparing the audio level pattern in the input signal from the microphone with the audio level pattern in the playback signal during the learning process. It calculates the correlation values for each tail position up to the configured echo tail length, and finds out which tail position has the best correlation value. This position is then marked as the echo position. While doing this, the ES also notes the gain factors, that is how much de-amplification to be applied to the input signal in order to remove the echo from the input. Once the tail position is found, the ES then applies the appropriate gain factor according to the state of the conversation (e.g. I’m talking you’re silent, you’re silent I’m talking, we’re both talking, etc.).

This works pretty well. On the downside though, we noticed that the ES still seems to let a quiet echo to pass and transmitted back to remote party, depending on the signal level at the time. For now we allow this to happen since the echo level is very low and  doesn’t seem to be too annoying, and lacking enough experience with the new algorithm we don’t want to make it cut signals too aggressively yet since it may inadvertently cut the “good” signal.

The new echo suppressor is available in SVN as pjmedia/echo_suppress.c, and it will be included in the next release.

PJSIP 0.9 is Released: Audio Latency, TURN implementation, IPv6, G.722, and More

Finally, after months of delay, PJSIP version 0.9.0 is released. This has been the longest gap (8 months) between releases, and consequently it has the most modifications in it (there have been 100+ tickets done on this release).

Some of the new features in this release:

  • many improvements in the audio, to reduce audio latency, to have better compatibility with more target platforms (Windows Vista issues have been fixed, as well as sporadic error reports for ALSA), and to maintain the audio quality against impairments such as clock drifts, bursty sound device, and of course, packet loss. Compared to version 0.8, I think we’ve improved audio latency by few hundred milliseconds.
  • support for TURN-07 in PJNATH, either as standalone client/server library, standalone client/server application (for testing purposes), or integrated with ICE-19. Just as we were the first to release open source ICE library, I think this is also the first open source TURN implementation out there. Unfortunately we haven’t had time to update it to TURN-08 as this draft was released late during our QA phase, but we’ll update it as soon as possible.
  • fixed the ICE offer/answer rules.
  • support for IPv6.
  • support for Secure RTP (SRTP)
  • better support for Windows Mobile target. We have new and more usable sample application (PocketPJ) and GSM and Speex codec should now be available for this target.
  • better support for Symbian S60 target. There is a more thorough Symbian tutorial available, and GSM and Speex codec should now be available for this target too.
  • implementation of G.722 codec.
  • support for RTCP Extended Report (XR)
  • and many more.

For more information, start from PJSIP download page. Get it while it’s hot!

PJSIP on Symbian Phone Works

[pjsip] PJSIP on Symbian Phone Works!:

This announcement is a bit late than planned (I promised to finish the Symbian port before Jan 2008), but we thought we’d give this a more thorough testing before announcing it, hence the delay.

In summary, PJSIP works and tested on Symbian S60 3rd Ed phone. Everything should work, including all SIP features, sound, STUN, and ICE. And we’ve made a tutorial on how to build and debug PJSIP on target device.

There have been some major changes along the way:

  • Carbide is now the preferred IDE rather than CodeWarrior. This is because CW has been deprecated by Nokia anyway.
  • Support for DSO is no longer enabled by default. It’s still supported, but you’d have to edit the MMP files slightly to build DSO outputs.

So what’s left to be done is for someone to create a nice open source Symbian softphone GUI on top of it. 😉

For more info:

Open source SIP stack, media, STUN, and ICE for Symbian OS

Just yesterday I finished back porting the Symbian branch to the trunk, and I think it’s good to go.

It’s been a roller-coaster way, supporting Symbian. It’s not the most developer friendly OS to port your programs to (see Readers Write about Symbian, OS X, and the iPhone), but we knew that, and I felt that this should make a good challenge for PJLIB, to see if it lives to its extreme portability claim. So we first started the port on May 2006, created a Symbian branch based on 0.5.5.6, and estimated that the work will need couple of months work. It wasn’t long before we realized we needed more time, and we revised the target to September 2006. But we still missed the target anyway.

Only about two months later, on Nov 2006, where we really had all of the libraries ported (only sound device is missing). But by this time, this branch was lagging waay behind the trunk, so it will take significant efforts (and commitments) to bring the port into the trunk.

But finally we had gathered enough “motivations” to do this, few days back, and it’s here.

Symbian target is officially supported in the trunk. All libraries have been ported. All seems to be running fine. No more panics. No memory leaks. All is good to go. Sound device is still missing, unfortunately.

So what do we have for the Symbian port again? For those new to PJSIP projects, here’s all of them:

  • pjlib, our platform abstraction hero.
  • pjlib-util, an auxiliary library containing parts needed by upper layer libraries (things like text scanning, XML, DNS SRV resolution, and various encryption algorithm),
  • pjnath, a NAT helper library containing the latest STUN, TURN, and ICE,
  • pjmedia, the media stack,
  • pjsip, pjsip-ua, pjsip-simple, libraries containing the bunch of SIP features,
  • pjsua-lib, a library combining SIP, media, and DNS SRV/STUN/ICE into high level API, and
  • symbian_ua, a simple console based SIP user agent for Symbian, based on pjsua-lib.

Some screenshot? Sure:

SIP User Agent on Symbian Emulator

Screenshot of symbian_ua on S60 Emulator

It’s been fun programming on Symbian. Learning curve has been steep. The tools are not perfect. But now I think I’m relatively more comfortable with it, and it becomes just another target for coding.

For more information about using the Symbian port, please see Porting PJ to Symbian OS. The porting effort itself is tracked on this Symbian-Porting Trac page.

Have fun yourself!

Can you help find a mobile WiFi chipset/SMT with GPL or open source drivers?

I’ve been tracking OpenMoko for a while and noticed this call for help for wifi support.

Readers of embedded SIP SDK like pjsip and pjmedia are usually clued up on sourcing issues like these, so maybe any one of you can help?

Maybe Matt can help as he knows everybody in the open source world!

OpenMoko


Subscribe to blog updates

View Perry Ismangil's profile on LinkedIn