LaForge's home page (GSM/GPRS/UMTS/LTE Protocol related)

Notfallwarnung im Mobilfunknetz + Cell Broadcast (Teil 2)

Harald Welte — Mon, 19 Jul 2021 16:00:00 GMT

[excuse this German-language post, this is targeted at the current German public discourse]

Ein paar Ergänzungen zu meinem blog-post gestern.

Ich benutzt den generischen Begriff PWS statt SMSCB, weil SMSCB strikt genommen nur im 2G-System existiert, und nur ein Layer ist, der dort für Notfallalarmierung verwendet wird.

Zu Notfallwarn-Apps

Natürlich sind spezielle, nationale Deutsche Katastrophenschutz-Apps auch nützlich! Aber diese sollten allenfalls zusätzlich angeboten werden, nachdem man erstmal die grundlegende Alarmierung konform der relevanten internationalen (und auch EU-)Standards via Cell Broadcast / PWS realisiert. Man sagt ja auch nicht: Nachrichtensendungen braucht man im Radio nicht mehr, weil man die bereits im Fernsehen hat. Man will auf allen verfügbaren Kanälen senden, und zunächst jene mit möglichst universeller Reichweite und klaren technischen Vorteilen benutzen, bevor man dann zusätzlich auch auf anderen Kanälen alarmiert.

Wie sieht PWS für mich als Anwender aus

Hier scheint es größere Missverständnisse zu geben, wie das auf dem Telefon letztlich aussieht. Ist ja auch verständlich, hierzulande sieht man das nie, ausser man ist zufällig in einem Labor/Basttel-Netz z.B. auf einer CCC-Veranstaltung unterwegs, in der das Osmocom-Projekt mal solche Nachrichten versendet hat.

Die PWS (ETWS, CMAS, WEA, KPAS, EU-ALERT, ...) nachrichten werden vom Telefon empfangen, und dann je nach Konfiguration und Priorität behandelt. Für die USA ist im WEA vorgeschrieben, dass Alarme einer bestimmten Prioritatsklasse (z.B. der Presidential Level Alert) immer zwangsweise zur Anzeige gebracht werden und immer mit einem lauten sirenenartigen Alarmton einhergehen. Es ist sogar explizit verboten, dass der Anwender diese Alarme irgendwo ausstellen, stumm schalten o.ä. kann. Insofern spielt es keine Rolle, ob das Telefon gerade Lautlos gestellt ist, oder es nicht gerade unmittelbar bei mir ist.

Bei manchen Geräten werden die Warnungen sogar mittels einer text2speech-Engine laut über den Lautsprecher vorgelesen, nachdem der Alarmton erscheint. Ob das eine regulatorische Anforderung eines der nationalen System ist, weiss ich nicht - ich habe es jedenfalls bereits in manchen Fällen gesehen, als ich mittels Osmocom-Software solche Alarme in privaten Labornetzen versandt habe.

Noch ein paar technische Details

PWS-Nachrichten werden auch dann noch ausgestrahlt, wenn die Zelle ihre Netzanbindung verloren hat. Wenn also z.B. das Glasfaserkabel zum Kernnetz bereits weg ist, aber noch Strom da ist, werden bereits vorher vom CBC (Cell Broadcast Centre) an die Mobilfunkzelle übermittelte Warnungen entsprechend ihrer Gültigkeitsdauer weiter autonom von der Zelle ausgesendet Das ist wieder ein inhärenter technischer Vorteil, der niemals mit einer App erreichbar ist, weil diese erfordert dass das komplette Mobilfunknetz mit allen internen Verbindungen und dem Kernnetz sowie die Internetverbindung vom Netzbetreiber zum Server des App-Anbieters durchgehend funktioniert.
PWS-Nachrichten können zumindest technisch auch von Telefonen empfangen werden, die garnicht im Netz eingebucht sind, oder die keine SIM eingelegt haben. Ob dies in den Standards gefordert wird, und/oder ob dies die jeweilige Telefonsoftware das so umsetzt, weiss ich nicht und müsste man prüfen. Technisch liegt es nahe, ähnlich wie das Absetzen von Notrufen, das ja auch technisch in diesen Fällen möglich ist.

Zu den Kosten

Wenn - wie in der idealen Welt - das Vorhalten von Notfallalarmierung eine Vorgabe bereits zum Zeitpunkt der Lizenzvergabe für Funkfrequenzen gewesen wäre, wäre das alles einfach ganz lautlos von Anfang an immer unterstützt gewesen. Keiner hätte extra Geld investieren müssen, weil diese minimale technische Vorgabe dann ja bereits Teil der Ausschreibungen der Betreiber für den Einkauf ihres Equipments gewesen wäre. Zudem hatten wir ja bereits in der Vergangenheit Cell Brodacast in allen drei Deutschen Netzen, d.h. die Technik war mal [aus ganz andern Gründen] vorhanden aber wurde irgendwann weggespart.

Das jetzt nachträglich einzuführen heisst natürlich, dass es niemand eingeplant hat, und dass jeder beteiligte am Markt sich das vergolden lassen will. Die Hersteller freuen sich in etwa wie "Oh, Ihr wollt jetzt mehr als ihr damals beim Einkauf spezifiziert habt? Schön, dann schreiben wir mal ein Angebot".

Technisch ist das alles ein Klacks. Die komplette Entwicklung aller Bestandteile für PWS in 2G/3G/4G/5G würde ich auf einen niedrigen einmaligen sechsstelligen Betrag schätzen. Und das ist die einmalige Investition in der Entwicklung, welche dann über alle Geräte/Länder/Netze umgebrochen wird. Bei den Milliarden, die in Entwicklung und Anschaffung von Mobilfunktechnik investiert wird, ist das ein Witz.

Die Geräte wie Basisstationen aller relevanten Hersteller unterstützen natürlich von Haus aus PWS. Die bauen für Deutschland ja nicht andere Geräte, als jene, die in UK, NL, RO, US, ... verbaut werden. Der Markt ist international, die gleiche Technik steht überall.

Weil man jetzt zu spät ist, wird das natürlich von allen Seiten ausgenutzt. Jeder Basisstationshersteller wird die Hand aufhalten und sagen, das kostet jetzt pro Zelle X EUR im Jahr zusätzliche Lizenzgebühren. Und die Anbieter der zentralen Komponente CBC werden auch branchenüblich die Hand aufhalten, mit satten jährlichen Lizenzgebühren. Und die Consultants werden auch alle die Hand aufhalten, weil es gibt wieder etwas zu Integrieren, zu testen, ... Das CBC ist keine komplexe Technik. Wenn man das einmalig als Open Source entwickeln lässt, und in allen Netzen einsetzt, bekommt man es quasi zum Nulltarif. Aber das würde ja Voraussetzen, dass man sich wirklich mit der Technik befasst, versteht um welch simple Software es hier geht, und dass man mal andere Wege in der Beschaffung geht, als nur mal eben bei seinen existierenden 3 Lieferanten anzurufen, die sich dann eine goldene Nase verdienen wollen.

In der öffentlichen Diskussion wird von 20-40 Millionen EUR gesprochen. Das sind überzogene Forderungen der Marktteilnehmer, nichts sonst. Aber selbst wenn man der Meinung ist, dass man lieber das Geld zum Fenster hinauswerfen will, statt Open Source Alternativen zu [ver]suchen, dann ist auch diese Größenordnung etwas, dass im Vergleich zu den sonstigen Anschaffungs- und Betriebskosten eines Mobilfunknetzes verschwindend gering ist. Ganz zu schweigen von den Folgekosten im Bereich Bergung/Rettung, Personenschäden, etc. die sich dadurch mittelfristig bei Katastrophen einsparen lassen.

Oder anders betrachtet: Wenn sogar das wirtschaftlich viel schwächere Rumänien sich sowas leisten kann, dann wird es wohl auch die Bundesrepublik Deutschland stemmen können.

Notfallwarnung im Mobilfunknetz + Cell Broadcast

Harald Welte — Sun, 18 Jul 2021 16:00:00 GMT

[excuse this German-language post, this is targeted at the current German public discourse]

In mehrerern Gegenden Deutschlands gab es verheerende Hochwasser, und die Öffentlichkeit diskutiert deshalb mal wieder die gute alte Frage nach dem adäquaten Mittel der Alarmierung der Bevölkerung.

Es ist einfach nur ein gigantisches Trauerspiel, wie sehr die Deutsche Politik und Verwaltung in diesem Punkt inzwischen seit Jahrzehnten sämtliche relevanten Standards verpennt, und dann immer wieder öffentlich durch fachlich falsche und völlig uninformierte Aussagen auffällt.

Das Thema wurde vor dem aktuellen Hochwasser bereits letztes Jahr im Rahmen des sog. WarnTag öffentlich diskutiert. Auch hier von Seiten der öffentlichen Hand ausschliesslich mit falschen Aussagen, wie z.B. dass es bei Cell Broadcast Datenschutzprobleme gibt. Dabei ist Cell Broadcast die einzige Technologie, wo keine Rückmeldung des einzelnen Netzteilnehmers erfolgt, und das Netz nichtmal weiss, wer die Nachricht empfangen hat, und wo dieser Empfang stattgefunden hat. Ganz wie beim UKW-Radio.

Fakt ist, dass alle digitalen Mobilfunkstandards seit GSM/2G, d.h. seit 1991 die Möglichkeit mitbringen, effizient, schnell und datensparsam alle Nutzer (einer bestimmten geographischen Region) mit sogenannten broadcast Nachrichten zu informieren. Diese Technik, in GSM/2G genannt Cell Broacast (oder auch _SMSCB_), unterscheidet sich Grundlegend von allen anderen Kommunikationsformen im Mobilfunknetz, wie Anrufe und herkömmliche SMS (offiziell SMS-PP). Anrufe, SMS und auch mobile Paketdaten (Internet) werden immer für jeden Teilnehmer individuell auf ihm zugewiesenen Funkressourcen übermittelt. Diese Ressourcen sind beschränkt. Es können in keinem Mobilfunknetz der Welt alle Teilnehmer gleichzeitig telefonieren, oder gleichzeitig SMS empfangen.

Stattdessen benutzt Cell Broadcast - wie der Name bereits unmissverständlich klar macht - Einen broadcast, d.h. Rundsendemechanismus. Eine Nachricht wird einmal gesendet, benötigt also nur eine geteilte Ressource auf der Luftschnittstelle, und wird dann von allen Geräten im Empfangsbereich zeitgleich empfangen und dekodiert. Das ist wie UKW-Radio oder klassisches terrestrisches Fernsehen.

Cell Broadcast wurde bereits in den 1990er Jahren von Deutschen Netzbetreibern benutzt. Und zwar nicht für etwas lebensnotwendiges wie die Notfallsignalisierung, sondern für so banale Dinge wie die Liste jener Vorwahlen, zu denen gerade ein vergünstigter "wandernder Ortstarif" Besteht. Ja, sowas gab es mal bei Vodafone. Oder bei O2 wurden über lange Zeit (aus unbekannten Gründen) die GPS-Koordinaten der jeweiligen Basisstation als Cell Broadcast versendet.

In der folgenden (nun fast abgeschalteten) Mobilfunkgeneration 3G wurde Cell Broadcast leicht umbenannt als Service Area Broadcast beibehalten. Schliesslich gibt es ja Länder mit - anders als in Deutschland - funktionierender und kompetenter Regulierung des Telekommunikationsmarktes, und die langjährig bestehenden gesetzlichen Anforderungen solcher Länder zwingen die Netzbetreiber und auch die Ausrüster der Neztbetreiber, neue Mobilfunkstandards so zu entwickeln, dass die gesetzlichen Vorgaben bzgl. der Alarmierung der Bevölkerung im Notfall funktioniert.

Im Rahmen dieser Standardisierung haben eine Reihe von Ländern innerhalb der 3GPP-Standardisierung (zuständig für 2G, 3G, 4G, 5G) sogenannte Public Warning Systems (PWS) standardisiert. Zu diesen gehören z.B. das Japanische ETWAS (Earthquake and Tsunami Warning System), das Koreanische KPAS (Korean Public Alerting System), das US-Amerikanische WEA (Wireless Emergency Alerts, früher bekannt als CMAS) und auch das EU-ALERT mit den nationalen Implementationen NL-ALERT (Niederlande) und UK-ALERT (Großbritannien) sowie RO-ALERT (Rumänien).

Die zahlreichen Studien und Untersuchungen, die zur Gestaltung obiger Systeme und der internationalen Standards im Mobilfunk geführt haben, weisen auch nochmal nach, was sowieso vorher jedem Techniker offensichtlich erscheint: Eine schelle Alarmierung aller Teilnehmer (einer Region) kann nur über einen Broadcast-Mechanismus erfolgen. In Japan war die Zielvorgabe, die Alarmierung in Erdbebenfällen innerhalb von weniger als 4 Sekunden an die gesamte betroffene Bevölkerung zu übertragen. Und das ist mit PWS möglich!

Die relevanten PWS-Standards in 2G/3G/4G/5G bieten jede Menge nützliche Funktionen:

Benachrichtigung in bestimmten geographischen Regionen
Interoperable Schnittstellen, so dass Netzwerkelemente unterschiedlicher Hersteller miteinander kommunizieren
Konfigurierbare Benachrichtigungstexte, nicht nur in der primären Landessprache, sondern auch in mehreren anderen Sprachen, die dann automatisch je nach Spracheinstellung des Telefons wiedergegeben werden
Unterschiedliche Schweregrade von Alarmierungen
Übermittlung nicht nur im Broadcast, sondern auch im Unicast an jeden Teilnehmer, der gerade in einem Telefongespräch ist, und dessen Telefon gerade währenddessen aus technischen Gründen den Broadcast nicht empfangen würde
Unterschied zwischen Wiederholung einer Übertragung ohne Änderung des Inhalts und einer übertragung mit geändertem Inhalt

Es gibt also seit vielen Jahren internationale Standards, wie sämtliche heute eingesetzten Mobilfunktechniken zur schnellen, effizienten und datensparsamen Alarmierung der Bevölkerung eingesetzt werden können.

Es gibt zahlreiche Länder, die diese Systeme seit langem einsetzen. Das US-Amerikanische WEA wurde nach eigenen Angaben seit 2012 bereits mehr als 61.000 Mal benutzt, um Menschen vor Unwetter oder anderen Katastrophen zu warnen.

Sogar innerhalb der EU hat man das EU-ALERT System spezifiziert, welches weitgehend mit dem amerikanischen WEA identisch ist, und auf die gleichen Techniken aufbaut.

Und dann gibt es Länder wie Deutschland, die es seit genauso vielen Jahren vermissen lassen, durch Gesetze oder Vorschriften

die Netzbetreiber zum Betrieb dieser Broadcast-Technologien in ihrem Netz verpflichtet
die Netzbetreiber zur Bereitstellung von standardisierten Schnittstellen gegenüber den Behörden wie Zivilschutz / Katastrophenschutz zu verpflichten, so das diese selbständig über alle Netzbetreiber Warnungen versenden können
die Gerätehersteller z.B. über Vorschriften des FTEG (Gesetz über Funkanlagen und Telekommunikationsendeinrichtungen) zu Verpflichten, die PWS-Nachrichten anzuzeigen

In den USA, dem vermeintlich viel mehr dem Freien Markt und dem Kapitalismus anhängenden System ist all dies der Regulierungsbehörde FCC möglich. In Deutschland mit seiner sozialen Marktwirtschaft ist es anscheinend unmöglich, den Markt entsprechend zu regulieren. Eine solche Regulierung schafft man in Deutschland nur für wirklich wichtige Themen wie zur Durchsetzung der Bereitstellung von Schnittstellen für die Telekommunikationsüberwachung. Bei so irrelevanten Themen wie dem Katastrophenschutz und der Alarmierung der Bevölkerung braucht man den Markt nicht zu regulieren. Wenn die Netzbetreiber kein PWS anbieten wollen, dann ist das einfach so Gottgegeben, und man kann da ja nichts machen.

Falls jemand sich SMSCB und PWS technisch näher ansehen will: In 2019 haben wir im Osmocom-Projekt eine Open Source Implementation des kompletten Systems von BTS über BSC bis zum CBC, sowie der dazwischen befindlichen Protokolle wie CBSP vorgenommen. Dies wurde freundlicherweise durch den Prototype Fund mit EUR 35k finanziert. Ja, so günstig kann man die nötige Technik zumindest für eine einzelne Mobilfunkgeneration entwickeln...

Man kann also in einem selbst betriebenen Labor-Mobilfunknetz, welches auf Open Source Software basiert mehr in Punkt standardkonformer Notfallalarmierung, als die Deutsche Politik, Verwaltung und Netzbetreiber zusammen hinbekommen.

Wir haben in Deutschland Leute, die diese Standards in und auswendig kennen, sogar daran mitgearbeitet haben. Wir haben Entwickler, die diese Standards implementiert haben. Aber wir schaffen es nicht, das auch mal selbst praktisch zu benutzen - das überlassen wir lieber den anderen Ländern. Wir lassen lieber zuerst die ganze Katastrophenalarmierung mittels Sirenen vergammeln, machen den Netzbetreibern keine Vorgaben, entwicklen komische Apps, die Anwender extra installieren müssen, die prinzipbedingt nicht skalieren und beim Test (WarnTag) nicht funktionieren.

Was für eine Glanzleistung für den hochentwickelten Techhologie-Standort Deutschland.

Software Freedom Podcast #3 about Free Software mobile phone communication

Harald Welte — Thu, 19 Dec 2019 16:00:00 GMT

Recently I had the pleasure of being part of the 3rd incarnation of a new podcast series by the Free Software Foundation Europe: The Software Freedom Podcast.

In episode 3, Matthias and Bonnie of the FSFE are interviewing me about various high-level topics related to [the lack of] Free Software in cellular telephony, as well as some of the projects that I was involved in (Openmoko, Osmocom).

We've also touched the current mainstream / political debate about Huawei and 5G networks, where my position can only be summarized as: It doesn't matter much in which country the related proprietary software is being developed. What we need is Free / Open Source software that can be publicly audited, and we need a method by which the operator can ensure that a given version of that FOSS software is actually executing on his equipment.

Thanks to the FSFE for covering such underdeveloped areas of Free Software, and to use their podcast to distribute related information and ideas.

The sad state of voice support in cellular modems

Harald Welte — Fri, 01 Sep 2017 16:00:00 GMT

Cellular modems have existed for decades and come in many shapes and kinds. They contain the cellular baseband processor, RF frontend, protocol stack software and anything else required to communicate with a cellular network. Basically a phone without display or input.

During the last decade or so, the vast majority of cellular modems come as LGA modules, i.e. a small PCB with all components on the top side (and a shielding can), which has contact pads on the bottom so you can solder it onto your mainboard. You can obtain them from vendors such as Sierra Wireless, u-blox, Quectel, ZTE, Huawei, Telit, Gemalto, and many others.

In most cases, the vendors now also solder those modules to small adapter boards to offer the same product in mPCIe form-factor. Other modems are directly manufactured in mPCIe or NGFF aka m.2 form-factor.

As long as those modems were still 2G / 2.5G / 2.75G, the main interconnection with the host (often some embedded system) was a serial UART. The Audio input/output for voice calls was made available as analog signals, ready to connect a microphone and spekaer, as that's what the cellular chipsets were designed for in the smartphones. In the Openmoko phones we also interfaced the audio of the cellular modem in analog, exactly for that reason.

From 3G onwards, the primary interface towards the host is now USB, with the modem running as a USB device. If your laptop contains a cellular modem, you will see it show up in the lsusb output.

From that point onwards, it would have made a lot of sense to simply expose the audio also via USB. Simply offer a multi-function USB device that has both whatever virutal serial ports for AT commands and network device for IP, and add a USB Audio device to it. It would simply show up as a "USB sound card" to the host, with all standard drivers working as expected. Sadly, nobody seems to have implemented this, at least not in a supported production version of their product

Instead, what some modem vendors have implemented as an ugly hack is the transport of 8kHz 16bit PCM samples over one of the UARTs. See for example the Quectel UC-20 or the Simcom SIM7100 which implement such a method.

All the others ignore any acess to the audio stream from software to a large part. One wonders why that is. From a software and systems architecture perspective it would be super easy. Instead, what most vendors do, is to expose a digital PCM interface. This is suboptimal in many ways:

there is no mPCIe standard on which pins PCM should be exposed
no standard product (like laptop, router, ...) with mPCIe slot will have anything connected to those PCM pins

Furthermore, each manufacturer / modem seems to support a different subset of dialect of the PCM interface in terms of

voltage (almost all of them are 1.8V, while mPCIe signals normally are 3.3V logic level)
master/slave (almost all of them insist on being a clock master)
sample format (alaw/ulaw/linear)
clock/bit rate (mostly 2.048 MHz, but can be as low as 128kHz)
frame sync (mostly short frame sync that ends before the first bit of the sample)
endianness (mostly MSB first)
clock phase (mostly change signals at rising edge; sample at falling edge)

It's a real nightmare, when it could be so simple. If they implemented USB-Audio, you could plug a cellular modem into any board with a mPCIe slot and it would simply work. As they don't, you need a specially designed mainboard that implements exactly the specific dialect/version of PCM of the given modem.

By the way, the most "amazing" vendor seems to be u-blox. Their Modems support PCM audio, but only the solder-type version. They simply didn't route those signals to the mPCIe slot, making audio impossible to use when using a connectorized modem. How inconvenient.

Summary

If you want to access the audio signals of a cellular modem from software, then you either

have standard hardware and pick one very specific modem model and hope this is available sufficiently long during your application, or
build your own hardware implementing a PCM slave interface and then pick + choose your cellular modem

On the Osmocom mpcie-breakout board and the sysmocom QMOD board we have exposed the PCM related pins on 2.54mm headers to allow for some separate board to pick up that PCM and offer it to the host system. However, such separate board hasn't been developed so far.

Returning from TelcoSecDay 2017 / General Musings

Harald Welte — Tue, 21 Mar 2017 10:00:00 GMT

I'm just on my way back from the Telecom Security Day 2017 <https://www.troopers.de/troopers17/telco-sec-day/>, which is an invitation-only event about telecom security issues hosted by ERNW back-to-back with their Troopers 2017 <https://www.troopers.de/troopers17/> conference.

I've been presenting at TelcoSecDay in previous years and hence was again invited to join (as attendee). The event has really gained quite some traction. Where early on you could find lots of IT security / hacker crowds, the number of participants from the operator (and to smaller extent also equipment maker) industry has been growing.

The quality of talks was great, and I enjoyed meeting various familiar faces. It's just a pity that it's only a single day - plus I had to head back to Berlin still today so I had to skip the dinner + social event.

When attending events like this, and seeing the interesting hacks that people are working on, it pains me a bit that I haven't really been doing much security work in recent years. netfilter/iptables was at least somewhat security related. My work on OpenPCD / librfid was clearly RFID security oriented, as was the work on airprobe, OsmocomTETRA, or even the EasyCard payment system hack

I have the same feeling when attending Linux kernel development related events. I have very fond memories of working in both fields, and it was a lot of fun. Also, to be honest, I believe that the work in Linux kernel land and the general IT security research was/is appreciated much more than the endless months and years I'm now spending my time with improving and extending the Osmocom cellular infrastructure stack.

Beyond the appreciation, it's also the fact that both the IT security and the Linux kernel communities are much larger. There are more people to learn from and learn with, to engage in discussions and ping-pong ideas. In Osmocom, the community is too small (and I have the feeling, it's actually shrinking), and in many areas it rather seems like I am the "ultimate resource" to ask, whether about 3GPP specs or about Osmocom code structure. What I'm missing is the feeling of being part of a bigger community. So in essence, my current role in the "Open Source Cellular" corner can be a very lonely one.

But hey, I don't want to sound more depressed than I am, this was supposed to be a post about TelcoSecDay. It just happens that attending IT Security and/or Linux Kernel events makes me somewhat gloomy for the above-mentioned reasons.

Meanwhile, if you have some interesting projcets/ideas at the border between cellular protocols/systems and security, I'd of course love to hear if there's some way to get my hands dirty in that area again :)

Gory details of USIM authentication sequence numbers

Harald Welte — Tue, 07 Mar 2017 16:00:00 GMT

I always though I understood UMTS AKA (authentication and key agreement), including the re-synchronization procedure. It's been years since I wrote tools like osmo-sim-auth which you can use to perform UMTS AKA with a SIM card inserted into a PC reader, i.e. simulate what happens between the AUC (authentication center) in a network and the USIM card.

However, it is only now as the sysmocom team works on 3G support of the dedicated OsmoHLR (outside of OsmoNITB!), that I seem to understand all the nasty little details.

I always thought for re-synchronization it is sufficient to simply increment the SQN (sequence number). It turns out, it isn't as there is a MSB-portion called SEQ and a lower-bit portion called IND, used for some fancy array indexing scheme of buckets of highest-used-SEQ within that IND bucket.

If you're interested in all the dirty details and associated spec references (the always hide the important parts in some Annex) see the discussion between Neels and me in Osmocom redmine issue 1965.

Cellular re-broadcast over satellite

Harald Welte — Wed, 15 Feb 2017 16:00:00 GMT

I've recently attended a seminar that (among other topics) also covered RF interference hunting. The speaker was talking about various real-world cases of RF interference and illustrating them in detail.

Of course everyone who has any interest in RF or cellular will know about fundamental issues of radio frequency interference. To the biggest part, you have

cells of the same operator interfering with each other due to too frequent frequency re-use, adjacent channel interference, etc.
cells of different operators interfering with each other due to intermodulation products and the like
cells interfering with cable TV, terrestrial TV
DECT interfering with cells
cells or microwave links interfering with SAT-TV reception
all types of general EMC problems

But what the speaker of this seminar covered was actually a cellular base-station being re-broadcast all over Europe via a commercial satellite (!).

It is a well-known fact that most satellites in the sky are basically just "bent pipes", i.e. they consist of a RF receiver on one frequency, a mixer to shift the frequency, and a power amplifier. So basically whatever is sent up on one frequency to the satellite gets re-transmitted back down to earth on another frequency. This is abused by "satellite hijacking" or "transponder hijacking" and has been covered for decades in various publications.

Ok, but how does cellular relate to this? Well, apparently some people are running VSAT terminals (bi-directional satellite terminals) with improperly shielded or broken cables/connectors. In that case, the RF emitted from a nearby cellular base station leaks into that cable, and will get amplified + up-converted by the block up-converter of that VSAT terminal.

The bent-pipe satellite subsequently picks this signal up and re-transmits it all over its coverage area!

I've tried to find some public documents about this, an there's surprisingly little public information about this phenomenon.

However, I could find a slide set from SES, presented at a Satellite Interference Reduction Group: Identifying Rebroadcast (GSM)

It describes a surprisingly manual and low-tech approach at hunting down the source of the interference by using an old nokia net-monitor phone to display the MCC/MNC/LAC/CID of the cell. Even in 2011 there were already open source projects such as airprobe that could have done the job based on sampled IF data. And I'm not even starting to consider proprietary tools.

It should be relatively simple to have a SDR that you can tune to a given satellite transponder, and which then would look for any GSM/UMTS/LTE carrier within its spectrum and dump their identities in a fully automatic way.

But then, maybe it really doesn't happen all that often after all to rectify such a development...

Contribute to Osmocom 3.5G and receive a free femtocell

Harald Welte — Wed, 28 Dec 2016 17:00:00 GMT

In 2016, Osmocom gained initial 3.5G support with osmo-iuh and the Iu interface extensions of our libmsc and OsmoSGSN code. This means you can run your own small open source 3.5G cellular network for SMS, Voice and Data services.

However, the project needs more contributors: Become an active member in the Osmocom development community and get your nano3G femtocell for free.

I'm happy to announce that my company sysmocom hereby issues a call for proposals to the general public. Please describe in a short proposal how you would help us improving the Osmocom project if you were to receive one of those free femtocells.

Details of this proposal can be found at https://sysmocom.de/downloads/accelerate_3g5_cfp.pdf

Please contact mailto:accelerate3g5@sysmocom.de in case of any questions.

Accessing 3GPP specs in PDF format

Harald Welte — Thu, 15 Dec 2016 17:00:00 GMT

When you work with GSM/cellular systems, the definite resource are the specifications. They were originally released by ETSI, later by 3GPP.

The problem start with the fact that there are separate numbering schemes. Everyone in the cellular industry I know always uses the GSM/3GPP TS numbering scheme, i.e. something like 3GPP TS 44.008. However, ETSI assigns its own numbers to the specs, like ETSI TS 144008. Now in most cases, it is as simple s removing the '.' and prefixing the '1' in the beginning. However, that's not always true and there are exceptions such as 3GPP TS 01.01 mapping to ETSI TS 101855. To make things harder, there doesn't seem to be a machine-readable translation table between the spec numbers, but there's a website for spec number conversion at http://webapp.etsi.org/key/queryform.asp

When I started to work on GSM related topics somewhere between my work at Openmoko and the start of the OpenBSC project, I manually downloaded the PDF files of GSM specifications from the ETSI website. This was a cumbersome process, as you had to enter the spec number (e.g. TS 04.08) in a search window, look for the latest version in the search results, click on that and then click again for accessing the PDF file (rather than a proprietary Microsoft Word file).

At some point a poor girlfriend of mine was kind enough to do this manual process for each and every 3GPP spec, and then create a corresponding symbolic link so that you could type something like evince /spae/openmoko/gsm-specs/by_chapter/44.008.pdf into your command line and get instant access to the respective spec.

However, of course, this gets out of date over time, and by now almost a decade has passed without a systematic update of that archive.

To the rescue, 3GPP started at some long time ago to not only provide the obnoxious M$ Word DOC files, but have deep links to ETSI. So you could go to http://www.3gpp.org/DynaReport/44-series.htm and then click on 44.008, and one further click you had the desired PDF, served by ETSI (3GPP apparently never provided PDF files).

However, in their infinite wisdom, at some point in 2016 the 3GPP webmaster decided to remove those deep links. Rather than a nice long list of released versions of a given spec, http://www.3gpp.org/DynaReport/44008.htm now points to some crappy JavaScript tabbed page, where you can click on the version number and then get a ZIP file with a single Word DOC file inside. You can hardly male it any more inconvenient and cumbersome. The PDF links would open immediately in modern browsers built-in JavaScript PDF viewer or your favorite PDF viewer. Single click to the information you want. But no, the PDF links had to go and replaced with ZIP file downloads that you first need to extract, and then open in something like LibreOffice, taking ages to load the document, rendering it improperly in a word processor. I don't want to edit the spec, I want to read it, sigh.

So since the usability of this 3GPP specification resource had been artificially crippled, I was annoyed sufficiently well to come up with a solution:

first create a complete mirror of all ETSI TS (technical specifications) by using a recursive wget on http://www.etsi.org/deliver/etsi_ts/
then use a shell script that utilizes pdfgrep and awk to determine the 3GPP specification number (it is written in the title on the first page of the document) and creating a symlink. Now I have something like 44.008-4.0.0.pdf -> ts_144008v040000p.pdf

It's such a waste of resources to have to download all those files and then write a script using pdfgrep+awk to re-gain the same usability that the 3GPP chose to remove from their website. Now we can wait for ETSI to disable indexing/recursion on their server, and easy and quick spec access would be gone forever :/

Why does nobody care about efficiency these days?

If you're also an avid 3GPP spec reader, I'm publishing the rather trivial scripts used at http://git.osmocom.org/3gpp-etsi-pdf-links

If you have contacts to the 3GPP webmaster, please try to motivate them to reinstate the direct PDF links.

The IT security culture, hackers vs. industry consortia

Harald Welte — Tue, 06 Dec 2016 00:00:00 GMT

In a previous life I used to do a lot of IT security work, probably even at a time when most people had no idea what IT security actually is. I grew up with the Chaos Computer Club, as it was a great place to meet people with common interests, skills and ethics. People were hacking (aka 'doing security research') for fun, to grow their skills, to advance society, to point out corporate stupidities and to raise awareness about issues.

I've always shared any results worth noting with the general public. Whether it was in RFID security, on GSM security, TETRA security, etc.

Even more so, I always shared the tools, creating free software implementations of systems that - at that time - were very difficult to impossible to access unless you worked for the vendors of related device, who obviously had a different agenda then to disclose security concerns to the general public.

Publishing security related findings at related conferences can be interpreted in two ways:

On the one hand, presenting at a major event will add to your credibility and reputation. That's a nice byproduct, but that shouldn't be the primarily reason, unless you're some kind of a egocentric stage addict.

On the other hand, presenting findings or giving any kind of presentation or lecture at an event is a statement of support for that event. When I submit a presentation at a given event, I think carefully if that topic actually matches the event.

The reason that I didn't submit any talks in recent years at CCC events is not that I didn't do technically exciting stuff that I could talk about - or that I wouldn't have the reputation that would make people consider my submission in the programme committee. I just thought there was nothing in my work relevant enough to bother the CCC attendees with.

So when Holger 'zecke' Freyther and I chose to present about our recent journeys into exploring modern cellular modems at the annual Chaos Communications Congress, we did so because the CCC Congress is the right audience for this talk. We did so, because we think the people there are the kind of community of like-minded spirits that we would like to contribute to. Whom we would like to give something back, for the many years of excellent presentations and conversations had.

So far so good.

However, in 2016, something happened that I haven't seen yet in my 17 years of speaking at Free Software, Linux, IT Security and other conferences: A select industry group (in this case the GSMA) asking me out of the blue to give them the talk one month in advance at a private industry event.

I could hardly believe it. How could they? Who am I? Am I spending sleepless nights and non-existing spare time into security research of cellular modems to give a free presentation to corporate guys at a closed industry meeting? The same kind of industries that create the problems in the first place, and who don't get their act together in building secure devices that respect people's privacy? Certainly not. I spend sleepless nights of hacking because I want to share the results with my friends. To share it with people who have the same passion, whom I respect and trust. To help my fellow hackers to understand technology one step more.

If that kind of request to undermine the researcher/authors initial publication among friends is happening to me, I'm quite sure it must be happening to other speakers at the 33C3 or other events, too. And that makes me very sad. I think the initial publication is something that connects the speaker/author with his audience.

Let's hope the researchers/hackers/speakers have sufficiently strong ethics to refuse such requests. If certain findings are initially published at a certain conference, then that is the initial publication. Period. Sure, you can ask afterwards if an author wants to repeat the presentation (or a similar one) at other events. But pre-empting the initial publication? Certainly not with me.

I offered the GSMA that I could talk on the importance of having FOSS implementations of cellular protocol stacks as enabler for security research, but apparently this was not to their interest. Seems like all they wanted is an exclusive heads-up on work they neither commissioned or supported in any other way.

And btw, I don't think what Holger and I will present about is all that exciting in the first place. More or less the standard kind of security nightmares. By now we are all so numbed down by nobody considering security and/or privacy in design of IT systems, that is is hardly any news. IoT how it is done so far might very well be the doom of mankind. An unstoppable tsunami of insecure and privacy-invading devices, built on ever more complex technology with way too many security issues. We shall henceforth call IoT the Industry of Thoughtlessness.

EC-GSM-IoT: Enhanced Coverage GSM for IoT

Harald Welte — Sat, 23 Jul 2016 04:00:00 GMT

In private conversation, Holger mentioned EC-GSM-IoT to me, and I had to dig a bit into it. It was introduced in Release 13, but if you do a web search for it, you find surprisingly little information beyond press releases with absolutely zero information content and no "further reading".

The primary reason for this seems to be that the feature was called EC-EGPRS until the very late stages, when it was renamed for - believe it or not - marketing reasons.

So when searching for the right term, you actually find specification references and change requests in the 3GPP document archives.

I tried to get a very brief overview, and from what I could find, it is centered around GERAN extension in the following ways:

EC-EGPRS goal: Improve coverage by 20dB
- New single-burst coding schemes
- Blind Physical Layer Repetitions where bursts are repeated up to 28 times without feedback from remote end
  - transmitter maintains phase coherency
  - receiver uses processing gain (like incremental redundancy?)
- New logical channel types (EC-BCCH, EC-PCH, EC-AGC, EC-RACH, ...)
- New RLC/MAC layer messages for the EC-PDCH communication
Power Efficient Operation (PEO)
- Introduction of eDRX (extended DRX) to allow for PCH listening intervals from minutes up to a hour
- Relaxed Idle Mode: Important to camp on a cell, not best cell. Reduces neighbor cell monitoring requirements

In terms of required modifications to an existing GSM/EDGE implementation, there will be (at least):

changes to the PHY layer regarding new coding schemes, logical channels and burst scheduling / re-transmissions
changes to the RLC/MAC layer in the PCU to implement the new EC specific message types and procedures
changes to the BTS and BSC in terms of paging in eDRX

In case you're interested in more pointers on technical details, check out the links provided at https://osmocom.org/issues/1780

It remains to be seen how widely this will be adopted. Rolling this cange out on moderm base station hardware seems technicalyl simple - but it remains to be seen how many equipment makers implement it, and at what cost to the operators. But I think the key issue is whether or not the baseband chipset makers (Intel, Qualcomm, Mediatek, ...) will implement it anytime soon on the device side.

There are no plans on implementing any of this in the Osmocom stack as of now,but in case anyone was interested in working on this, feel free to contact us on the osmocom-net-gprs@lists.osmocom.org mailing list.

Deeper ventures into Ericsson (Packet) Abis

Harald Welte — Sat, 16 Jul 2016 04:00:00 GMT

Some topics keep coming back, even a number of years after first having worked on them. And then you start to search online using your favorite search engine - and find your old posts on that subject are the most comprehensive publicly available information on the subject ;)

Back in 2011, I was working on some very basic support for Ericsson RBS2xxx GSM BTSs in OpenBSC. The major part of this was to find out the weird dynamic detection of the signalling timeslot, as well as the fully non-standard OM2000 protocol for OML. Once it reached the state of a 'proof-of-concept', work at this ceased and remained in a state where still lots of manual steps were involved in BTS bring-up.

I've recently picked this topic up again, resulting in some work-in-progress code in http://git.osmocom.org/openbsc/log/?h=laforge/om2000-fsm

Beyond classic E1 based A-bis support, I've also been looking (again) at Ericsson Packet Abis. Packet Abis is their understanding of Abis over IP. However, it is - again - much further from the 3GPP specifications than what we're used to in the Osmocom universe. Abis/IP as we know consists of:

RSL and OML over TCP (inside an IPA multiplex)
RTP streams for the user plane (voice)
Gb over IP (NS over UDP/IP), as te PCU is in the BTS.

In the Ericsson world, they decided to taka a much lower-layer approach and decided to

start with L2TP over IP (not the L2TP over UDP that many people know from VPNs)
use the IETF-standardized Pseudowire type for HDLC but use a frame format in violation of the IETF RFCs
Talk LAPD over L2TP for RSL and OML
Invent a new frame format for voice codec frames called TFP and feed that over L2TP
Invent a new frame format for the PCU-CCU communication called P-GSL and feed that over L2TP

I'm not yet sure if we want to fully support that protocol stack from OpenBSC and related projects, but in any case I've extende wireshark to decode such protocol traces properly by

Extending the L2TP dissector with Ericsson specific AVPs
Improving my earlier pakcet-ehdlc.c with better understanding of the protocol
Implementing a new TFP dissector from scratch
Implementing a new P-GSL dissector from scratch

The resulting work can be found at http://git.osmocom.org/wireshark/log/?h=laforge/ericsson-packet-abis in case anyone is interested. I've mostly been working with protocol traces from RBS2409 so far, and they are decoded quite nicely for RSL, OML, Voice and Packet data. As far as I know, the format of the STN / SIU of other BTS models is identical.

Is anyone out there in possession of Ericsson RBS2xxx RBSs interested in collboration on either a Packet Abis implementation, or an inteface of the E1 or packet based CCU-PCU interface to OsmoPCU?

Software under OSA Public License is neither Open Source nor Free Software

Harald Welte — Tue, 23 Feb 2016 16:00:00 GMT

It seems my recent concerns on the OpenAirInterface re-licensing were not unjustified.

I contacted various legal experts on Free Software legal community about this, and the response was unanimous: In all feedback I received, the general opinion was that software under the OSA Public License V1.0 is neither Free Software nor Open Source Software.

The rational is, that it does not fulfill the criteria of

the FSF Free Software definition, as the license does not fulfill freedom 0: The freedom to run the program as you wish, for any purpose (which obviously includes commercial use)
the Open Source Initiatives Open Source Definition, as the license must not discriminate against fields of endeavor, such as commercial use.
the Debian Free Software Guidelines, as the DFSG also require no discrimination against fields of endeavor, such as commercial use.

I think we as the community need to be very clear about this. We should not easily tolerate that people put software under restrictive licenses but still call that software open source. This creates a bad impression to those not familiar with the culture and spirit of both Free Software and Open Source. It creates the impression that people can call something Open Source but then still ask royalties for it, if used commercially.

It is a shame that entities like Eurecom and the OpenAirInterface Software Association are open-washing their software by calling it Open Source when in fact it isn't. This attitude frankly makes me sick.

That's just like green-washing when companies like BP are claiming they're now an environmental friendly company just because they put some solar panels on the roof of some building.

On the OpenAirInterface re-licensing

Harald Welte — Sun, 31 Jan 2016 16:00:00 GMT

In the recent FOSDEM 2016 SDR Devroom, the Q&A session following a presentation on OpenAirInterface touched the topic of its controversial licensing. As I happen to be involved deeply with Free Software licensing and Free Software telecom topics, I thought I might have some things to say about this topic. Unfortunately the Q&A session was short, hence this blog post.

As a side note, the presentation was actually certainly the least technical presentation in all of the FOSDEM SDR track, and that with a deeply technical audience. And probably the only presentation at all at FOSDEM talking a lot about "Strategic Industry Partners".

Let me also state that I actually have respect for what OAI/OSA has been and still is doing. I just don't think it is attractive to the Free Software community - and it might actually not be Free Software at all.

OpenAirInterface / History

Within EURECOM, a group around Prof. Raymond Knopp has been working on a Free Software implementation of all layers of the LTE (4G) system known as OpenAirInterface. It includes the physical layer and goes through to the core network.

The OpenAirInterface code was for many years under GPL license (GPLv2, other parts GPLv3). Initially the SVN repositories were not public (despite the license), but after some friendly mails one (at least I) could get access.

I've read through the code at several points in the past, it often seemed much more like a (quick and dirty?) proof of concept implementation to me, than anything more general-purpose. But then, that might have been a wrong impression on my behalf, or it might be that this was simply sufficient for the kind of research they wanted to do. After all, scientific research and FOSS often have a complicated relationship. Researchers naturally have their papers as primary output of their work, and software implementations often are more like a necessary evil than the actual goal. But then, I digress.

Now at some point in 2014, a new organization the OpenAirInterface Software Association (OSA) was established. The idea apparently was to get involved with the tier-1 telecom suppliers (like Alcatel, Huawei, Ericsson, ...) and work together on an implementation of Free Software for future mobile data, so-called 5G technologies.

Telecom Industry and Patents

In case you don't know, the classic telecom industry loves patents. Pretty much anything and everything is patented, and the patents are heavily enforced. And not just between Samsung and Apple, or more recently also Nokia and Samsung - but basically all the time.

One of the big reasons why even the most simple UMTS/3G capable phones are so much more expensive than GSM/2G is the extensive (and expensive) list of patents Qualcomm requires every device maker to license. In the past, this was not even a fixed per-unit royalty, but the license depended on the actual overall price of the phone itself.

So wanting to work on a Free Software implementation of future telecom standards with active support and involvement of the telecom industry obviously means contention in terms of patents.

Re-Licensing

The existing GPLv2/GPLv3 license of the OpenAirInterface code of course would have meant that contributions from the patent-holding telecom industry would have to come with appropriate royalty-free patent licenses. After all, of what use is it if the software is free in terms of copyright licensing, but then you still have the patents that make it non-free.

Now the big industry of course wouldn't want to do that, so the OSA decided to re-license the code-base under a new license.

As we apparently don't yet have sufficient existing Free Software licenses, they decided to create a new license. That new license (the OSA Public License V1.0 not only does away with copyleft, but also does away with a normal patent grant.

This is very sad in several ways:

license proliferation is always bad. Major experts and basically all major entities in the Free Software world (FSF, FSFE, OSI, ...) are opposed to it and see it as a problem. Even companies like Intel and Google have publicly raised concern about license Proliferation.
abandoning copyleft. Many people particularly from a GNU/Linux background would agree that copyleft is a fair deal. It ensures that everyone modifying the software will have to share such modifications with other users in a fair way. Nobody can create proprietary derivatives.
taking away the patent grant. Even the non-copyleft Apache 2.0 License the OSA used as template has a broad patent grant, even for commercial applications. The OSA Public License has only a patent grant for use in research context

In addition to this license change, the OSA also requires a copyright assignment from all contributors.

Consequences

What kind of effect does this have in case I want to contribute?

I have to sign away my copyright. The OSA can at any given point in time grant anyone whatever license they want to this code.
I have to agree to a permissive license without copyleft, i.e. everyone else can create proprietary derivatives of my work
I do not even get a patent grant from the other contributors (like the large Telecom companies).

So basically, I have to sign away my copyright, and I get nothing in return. No copyleft that ensures other people's modifications will be available under the same license, no patent grant, and I don't even keep my own copyright to be able to veto any future license changes.

My personal opinion (and apparently those of other FOSDEM attendees) is thus that the OAI / OSA invitation to contributions from the community is not a very attractive one. It might all be well and fine for large industry and research institutes. But I don't think the Free Software community has much to gain in all of this.

Now OSA will claim that the above is not true, and that all contributors (including the Telecom vendors) have agreed to license their patents under FRAND conditions to all other contributors. It even seemed to me that the speaker at FOSDEM believed this was something positive in any way. I can only laugh at that ;)

FRAND

FRAND (Fair, Reasonable and Non-Discriminatory) is a frequently invoked buzzword for patent licensing schemes. It isn't actually defined anywhere, and is most likely just meant to sound nice to people who don't understand what it really means. Like, let's say, political decision makers.

In practise, it is a disaster for individuals and small/medium sized companies. I can tell you first hand from having tried to obtain patent licenses from FRAND schemes before. While they might have reasonable per-unit royalties and they might offer those royalties to everyone, they typically come with ridiculous minimum annual fees.

For example let's say they state in their FRAND license conditions you have to pay 1 USD per device, but a minimum of USD 100,000 per year. Or a similarly large one-time fee at the time of signing the contract.

That's of course very fair to the large corporations, but it makes it impossible for a small company who sells maybe 10 to 100 devices per year, as the 100,000 / 10 then equals to USD 10k per device in terms of royalties. Does that sound fair and Non-Discriminatory to you?

Summary

OAI/OSA are trying to get a non-commercial / research-oriented foot into the design and specification process of future mobile telecom network standardization. That's a big and difficult challenge.

However, the decisions they have taken in terms of licensing show that they are primarily interested in aligning with the large corporate telecom industry, and have thus created something that isn't really Free Software (missing non-research patent grant) and might in the end only help the large telecom vendors to uni-directionally consume contributions from academic research, small/medium sized companies and individual hackers.

32C3 is over, GSM and GPRS was running fine, osmo-iuh progress

Harald Welte — Wed, 30 Dec 2015 16:00:00 GMT

The 32C3 GSM Network

32C3 was great from the Osmocom perspective: We could again run our own cellular network at the event in order to perform load testing with real users. We had 7 BTSs running, each with a single TRX. What was new compared to previous years:

OsmoPCU is significantly more robust and stable due to the efforts of Jacob Erlbeck at sysmocom. This means that GPRS is now actually still usable in severe overload situations, like 1000 subscribers sharing only very few kilobits. Of course it will be slow, but at least data still passes through as much as that's possible.
We were using half-rate traffic channels from day 2 onwards, in order to enhance capacity. Phones supporting AMR-HR would use that, but then there are lots of old phones that only do classic HR (v1). OsmoNITB with internal MNCC handler supports TCH/H with HR and AMR for at least five years, but the particular combination of OsmoBTS + OsmoNITB + lcr (all master branches) was not yet deployed at previous CCC event networks so far.

Being forced to provide classic HR codec actually revealed several bugs in the existing code:

OsmoBTS (at least with the sysmoBTS hardware) is using bit ordering that is not compliant to what the spec says on how GSM-HR frames should be put into RTP frames. We didn't realize this so far, as handing frames from one sysmoBTS to another sysmoBTS of course works, as both use the same (wrong) bit ordering.
The ETSI reference implementation of the HR codec has lots of global/static variables, and thus doesn't really support running multiple transcoders in parallel. This is however what lcr was trying (and needing) to do, and it of course failed as state from one transcoder instance was leaking into another. The problem is simple, but the solution not so simple. If you want to avoid re-structuring the entire code in very intrusive ways or running one thread per transcoder instance, then the only solution was to basically memcpy() the entire data section of the transcoding library every time you switch the state from one transcoder instance to the other. It's surprisingly difficult to learn the start + size of that data section at runtime in a portable way, though.

Thanks to our resident voice codec expert Sylvain for debugging and fixing the above two problems.

Thanks also to Daniel and Ulli for taking care of the actual logistics of bringing + installing (+ later unmounting) all associated equipment.

Thanks furthermore to Kevin who has been patiently handling the 'Level 2 Support' cases of people with various problems ending up in the GSM room.

It's great that there is a team taking care of those real-world test networks. We learn a lot more about our software under heavy load situations this way.

osmo-iuh progress + talk

I've been focussing basically full day (and night) over the week ahead of Christmas and during Christmas to bring the osmo-iuh code into a state where we could do a end-to-end demo with a regular phone + hNodeB + osmo-hnbgw + osmo-sgsn + openggsn. Unfortunately I only got it up to the point where we do the PDP CONTEXT ACTIVATION on the signalling plane, with no actual user data going back and forth. And then, for strange reasons, I couldn't even demo that at the end of the talk. Well, in either case, the code has made much progress.

The video of the talk can be found at https://media.ccc.de/v/32c3-7412-running_your_own_3g_3_5g_network#video

meeting friends

The annual CCC congress is always an event where you meet old friends and colleagues. It was great talking to Stefan, Dimitri, Kevin, Nico, Sylvain, Jochen, Sec, Schneider, bunnie and many other hackers. After the event is over, I wish I could continue working together with all those folks the rest of the year, too :/

Some people have been missed dearly. Absence from the CCC congress is not acceptable. You know who you are, if you're reading this ;)

Anyone interested in supporting SMPP interworking at 32C3?

Harald Welte — Sat, 05 Dec 2015 16:00:00 GMT

Sylvain brought this up yesterday: Wouldn't it be nice to have some degree of SMS interfacing from OpenBSC/OsmoNITB to the real world at 32C3? It is something that we've never tried so far, and thus definitely worthy of testing.

Of course, full interworking is not possible without assigning public MSISDN to all internal subscribers / 'extensions' how we call them.

But what would most certainly work is to have at least outbound SMS working by means of an external SMPP interface.

The OsmoNITB-internal SMSC speaks SMPP already (in the SMSC role), so we would need to implement some small amount of glue logic that behaves as ESME (external SMS entity) towards both OsmoNITB as well as some public SMS operator/reseller that speaks SMPP again.

Now of course, sending SMS to public operators doesn't come for free. So in case anyone reading this has access to SMPP at public operators, resellers, SMS hubs, it would be interesting to see if there is a chance for some funding/sponsoring of that experiment.

Feel free to contact me if you see a way to make this happen.

python-libsmpp works great with OsmoNITB

Harald Welte — Fri, 04 Dec 2015 16:00:00 GMT

Since 2012 we have support for SMPP in OsmoNITB (the network-in-the-box version of OpenBSC). So far I've only used it from C and Erlang code.

Yesterday I gave python-smpplib from https://github.com/podshumok/python-smpplib a try and it worked like a charm. Of course one has to get the details right (like numbering plan indication).

In case anyone is interested in interfacing OsmoNITB SMPP from python, I've put a working example to send SMS at http://cgit.osmocom.org/mncc-python/tree/smpp_test.py

Python tool to talk to OsmoNITB MNCC interface

Harald Welte — Tue, 01 Dec 2015 16:00:00 GMT

I've been working on a small python tool that can be used to attach to the MNCC interface of OsmoNITB. It implements the 04.08 CC state machine with our MNCC primitives, including support for RTP bridge mode of the voice streams.

The immediate first use case for this was to be able to generate MT calls to a set of known MSISDNs and load all 14 TCH/H channels of a single-TRX BTS. It will connect the MT calls in pairs, so you end up with 7 MS-to-MS calls.

The first working version of the tool is available from

http://git.osmocom.org/mncc-python/
git clone git://git.osmocom.org/mncc-python

The code is pretty hacky in some places. That's partially due to the fact that I'm much more familiar in the C, Perl and Erlang world than in python. Still I thought it's a good idea to do it in python to enable more people to use/edit/contribute to it.

I'm happy for review / cleanup suggestion by people with more Python-foo than I have.

Architecturally, I decided to do things a bit erlang-like, where we have finite state machines in an actor models, and message passing between the actors. This is what happens with the GsmCallFsm()'s, which are created by the GsmCallConnector() representing both legs of a call and the MnccActor() that wraps the MNCC socket towards OsmoNITB.

The actual encoding/decoding of MNCC messages is auto-generated from the mncc header file #defines, enums and c-structures by means of ctypes code generation.

mncc_test.py currently drops you into a python shell where you can e.g. start more / new calls by calling functions like connect_call("7839", "3802") from that shell. Exiting the shell by quit() or Ctrl+C will terminate all call FSMs and terminate.

GSM test network at 32C3, after all

Harald Welte — Sun, 15 Nov 2015 16:00:00 GMT

Contrary to my blog post yesterday, it looks like we will have a private GSM network at the CCC congress again, after all.

It appears that Vodafone Germany (who was awarded the former DECT guard band in the 2015 spectrum auctions) is not yet using it in December, and they agreed that we can use it at the 32C3.

With this approval from Vodafone Germany we can now go to the regulator (BNetzA) and obtain the usual test license. Given that we used to get the license in the past, and that Vodafone has agreed, this should be a mere formality.

For the German language readers who appreciate the language of the administration, it will be a Frequenzzuteilung für Versuchszwecke im nichtöffentlichen mobilen Landfunk.

So thanks to Vodafone Germany, who enabled us at least this time to run a network again. By end of 2016 you can be sure they will have put their new spectrum to use, so I'm not that optimistic that this would be possible again.

No GSM test network at 32C3

Harald Welte — Sat, 14 Nov 2015 16:00:00 GMT

I currently don't assume that there will be a GSM network at the 32C3.

Ever since OpenBSC was created in 2008, the annual CCC congress was a great opportunity to test OpenBSC and related software with thousands of willing participants. In order to do so, we obtained a test licence from the German regulatory authority. This was never any problem, as there was a chunk of spectrum in the 1800 MHz GSM band that was not allocated to any commercial operator, the so-called DECT guard band. It's called that way as it was kept free in order to ensure there is no interference between 1800 MHz GSM and the neighboring DECT cordless telephones.

Over the decades, it was determined on a EU level that this guard band might not be necessary, or at least not if certain considerations are taken for BTSs deployed in that band.

When the German regulatory authority re-auctioned the GSM spectrum earlier this year, they decided to also auction the frequencies of the former DECT guard band. The DECT guard band was awarded to Vodafone.

This is a pity, as this means that people involved with cellular research or development of cellular technology now have it significantly harder to actually test their systems.

In some other EU member states it is easier, like in the Netherlands or the UK, where the DECT guard band was not treated like any other chunk of the GSM bands, but put under special rules. Not so in Germany.

To make a long story short: Without the explicit permission of any of the commercial mobile operators, it is not possible to run a test/experimental network like we used to ran at the annual CCC congress.

Given that

the event is held in the city center (where frequencies are typically used and re-used quite densely), and
an operator has nothing to gain from permitting us to test our open source GSM/GPRS implementations,

I think there is little chance that this will become a reality.

If anyone has really good contacts to the radio network planning team of a German mobile operator and wants to prove me wrong: Feel free to contact me by e-mail.

Thanks to everyone involved with the GSM team at the CCC events, particularly Holger Freyther, Daniel Willmann, Stefan Schmidt, Jan Luebbe, Peter Stuge, Sylvain Munaut, Kevin Redon, Andreas Eversberg, Ulli (and everyone else whom I may have forgot, my apologies). It's been a pleasure!

Thanks also to our friends at the POC (Phone Operation Center) who have provided interfacing to the DECT, ISDN, analog and VoIP network at the events. Thanks to roh for helping with our special patch requests. Thanks also to those entities and people who borrowed equipment (like BTSs) in the pre-sysmocom years.

So long, and thanks for all the fish!

Osmocom Berlin meetings

Harald Welte — Sat, 07 Nov 2015 16:00:00 GMT

Back in 2012, I started the idea of having a regular, bi-weekly meeting of people interested in mobile communications technology, not only strictly related to the Osmocom projects and software. This was initially called the Osmocom User Group Berlin. The meetings were held twice per month in the rooms of the Chaos Computer Club Berlin.

There are plenty of people that were or still are involved with Osmocom one way or another in Berlin. Think of zecke, alphaone, 2b-as, kevin, nion, max, prom, dexter, myself - just to name a few.

Over the years, I got "too busy" and was no longer able to attend regularly. Some people kept it alive (thanks to dexter!), but eventually they were discontinued in 2013.

Starting in October 2015, I started a revival of the meetings, two have been held already, the third is coming up next week on November 11.

I'm happy that I had the idea of re-starting the meeting. It's good to meet old friends and new people alike. Both times there actually were some new faces around, most of which even had a classic professional telecom background.

In order to emphasize the focus is strictly not on Osmocom alone ( particularly not about its users only), I decided to rename the event to the Osmocom Meeting Berlin.

If you're in Berlin and are interested in mobile communications technology on the protocol and radio side of things, feel free to join us next Wednesday.

Progress on the Linux kernel GTP code

Harald Welte — Sat, 07 Nov 2015 16:00:00 GMT

It is always sad if you start to develop some project and then never get around finishing it, as there are too many things to take care in parallel. But then, days only have 24 hours...

Back in 2012 I started to write some generic Linux kernel GTP tunneling code. GTP is the GPRS Tunneling Protocol, a protocol between core network elements in GPRS networks, later extended to be used in UMTS and even LTE networks.

GTP is split in a control plane for management and the user plane carrying the actual user IP traffic of a mobile subscriber. So if you're reading this blog via a cellular interent connection, your data is carried in GTP-U within the cellular core network.

To me as a former Linux kernel networking developer, the user plane of GTP (GTP-U) had always belonged into kernel space. It is a tunneling protocol not too different from many other tunneling protocols that already exist (GRE, IPIP, L2TP, PPP, ...) and for the user plane, all it does is basically add a header in one direction and remove the header in the other direction. User data, particularly in networks with many subscribers and/or high bandwidth use.

Also, unlike many other telecom / cellular protocols, GTP is an IP-only protocol with no E1, Frame Relay or ATM legacy. It also has nothing to do with SS7, nor does it use ASN.1 syntax and/or some exotic encoding rules. In summary, it is nothing like any other GSM/3GPP protocol, and looks much more of what you're used from the IETF/Internet world.

Unfortunately I didn't get very far with my code back in 2012, but luckily Pablo Neira (one of my colleagues from netfilter/iptables days) picked it up and brought it along. However, for some time it has been stalled until recently it was thankfully picked up by Andreas Schultz and now receives some attention and discussion, with the clear intention to finish + submit it for mainline inclusion.

The code is now kept in a git repository at http://git.osmocom.org/osmo-gtp-kernel/

Thanks to Pablo and Andreas for picking this up, let's hope this is the last coding sprint before it goes mainline and gets actually used in production.

Attending HITCON and COSCUP in Taipei

Harald Welte — Tue, 04 Jun 2013 19:00:00 GMT

It is my pleasure to attend the HITCON 2013 and COSCUP 2013 conferences in July/August this year. They are both in Taipei. HITCON is a hacker/security event, while COSCUP is a pure Free/Open Source Software conference.

At both events I will be speaking at the growing list of GSM related tools that are available these days, like OpenBSC, OsmcoomBB, SIMtrace, OsmoSGSN, OsmoBTS, OsmoSDR, etc. As they are both FOSS projects and useful in a security context, this fits well within the scope of both events.

Given that I'm going to be back to Taiwan, I'm looking very much forward to meeting old friends and former colleagues from my Openmoko days in Taipei. God, do I miss those days. While terribly stressful, they still are the most exciting days of my career so far.

And yes, I'm also going to use the opportunity for a continuation of my motorbike riding in this beautiful country.

Inside a cavity duplexer

Harald Welte — Wed, 21 Nov 2012 19:00:00 GMT

In many cellular systems (GSM or otherwise) there is a frequency duplex between the uplink and downlink frequency band. If you use a single antenna to serve a BTS, then somehow you need to split the frequency band between the Rx and Tx side by means of a Duplexer.

The most common technology for this is the so-called Cavity Duplexer. I've used those devices (and seen them in use) for a long time, but never really opened one so far. The problem is that they are finely tuned, and each mechanical change can severely impact performance. As I had to repair a broken SMA socket on one of them recently, I took the chance to take a picture

In the first picture you can see the bottom side. This consists of a milled aluminum block, with a series of circular cavities. The Tx output of the BTS is connected to the SMA socket on the bottom right, the antenna to the SMA socket on the top side, and the Rx port to the SMA socket on the bottom left of the picture:

The small cylindrical objects in the center of the cavities are not milled from the same part, but they are separate pieces mounted by screws from the bottom of the unit.

The second picture shows the top section of the duplexer:

You can see a ~ 4mm aluminum plate with lots of (now empty) holes which are for the ~ 117 screws with which the top plate is screwed against the bottom part shown in the first picture.

The important part, however, are the screws that you can see sticking out of the top part. Those are used for tuning and present "obstacles" in the path of the waves as they pass through the cavities.

The big miracle for me is not that there are some resonances which build up a filter, but that you can actually transfer as much as 100W of RF power from the Tx input through to the antenna output.

Kevin Redon starts collaborative Osmocom project to collect terminal profile

Harald Welte — Sun, 20 May 2012 19:00:00 GMT

As Kevin Redon writes in his blog, he has created some tools and a project for collaboratively gathering a database on the TERMINAL PROFILE capabilities of mobile phones.

The terminal profile describes which particular features regarding proactive sim or sim application toolkit a given phone supports.

This is not only important for SIM application / SIM toolkit developers, but it is also an important factor when trying to analyze the potential threat that can originate from a malicious SIM card attack.

I personally see no reason why my phone should ever report its GPS position to the SIM card, or why the SIM card should be able to re-write the nubers I'm dialling. Yes, there are cases where such features are useful, but then they should be explicitly enabled by the user, and the default should be that they are all switched off.

Who knows, after all, with some attention to this problem we might still see a SIM firewall / proxy, that you can put between the SIM and the phone to prevent any of those features from being (mis)used.

So all you need to do to contribute to the database is some way how you can read out the terminal profile from your mobile phone(s), and use Kevin's tool to upload it to the public website. And hwo do you read out the terminal profile? For example by using Osmocom SIMtrace to sniff the communication between SIM card and phone.

Announcing the low-power, light-weight sysmoBTS

Harald Welte — Fri, 18 May 2012 19:00:00 GMT

It hasn't been a secret that when I co-started a company called sysmocom more than a year ago, it was not about opening a webshop that sells cheap phones and DYI electronics kits to the larger community. Rather, it was to develop and sell exciting products surrounding Free Software and mobile communications.

There are of course the more or less obvious things to do, like system integration of OpenBSC and the related software on embedded systems, selling them as appliances including training, support and maintenance service.

However, we of course also want to more than that. Today it is my pleasure to say that the availability of our first BTS product called sysmoBTS has been officially announced.

See the news item, the product page and the data sheet for more information.

To make it very clear in the beginning: sysmoBTS is not an open hardware project. The schematics and layout files are proprietary and not disclosed publicly. Such is the FPGA bitstream and the layer1 inside the DSP.

However, any code running on the integrated ARM processor is available as free software. This includes a yocto/poky-built Embedded Linux distribution featuring u-boot, the Linux kernel (including all kernel modules!), the osmo-bts and OpenBSC software as well as many other Free Software packages.

We think this is a reasonable compromise between espanding a bit from our previous "BSC and above in Free Software" down to a "BTS Layer2 and above" divide. After all, if you use OpenBSC with a BTS from Siemens, Ericsson, Nokia or ip.access, you don't have access to the source code of anything running inside the BTS at all.

sysmoBTS offers some great new capabilities, such as integrating the BSC or even the entire osmo-nitb onto the ARM/Linux processor inside the BTS hardware itself, creating a less than 500gram, 10W power consuming autonomous GSM network.

I'm going to stop marketing here, but I thought it is one of the major milestones for sysmoocm and thus for what I've spent way too much time on in recent months - and thus deserves to be mentioned here on this personal blog.

Alcatel MTK phone UART pinout

Harald Welte — Tue, 13 Mar 2012 19:00:00 GMT

The Alcatel OT-890D is a MT6573 based smartphone. It seems one of the UARTs is available on test pads as seen in this picture:

The voltage level is still 3.3V, so no fancy 1.8V gear is required.

During boot, the UART is first used at 19200 bps, where it prints the strings "MW01" and "MW02". I then switches to 115200 bps where it prints "READY", and finally switches to 921600 bps, where it seems to output some mixed binary/text messages containing AT commands and responses between AP and BP, as well as some debug information:

�Ue� � � T+CREG=2
�Ue�!�!�!T+CSQSQ=1
�Ue�!�!�!AT+CREG=2
�Uew�"w�"w�"SQSQ=1
'Ue"""      AT+EFUN=1
      SML: Load!_Ue""""""
                         SML: Load!hU("("("

I haven't yet investigated if the binary between the text is some standard HDLC framing or a TS 07.10 multiplex.

If anyone knows more about the boot process (MW01/MW02/READY) or the binary protocol, please let me know.

More research into the Motorola Horizon macro and Mo-bis

Harald Welte — Wed, 29 Feb 2012 19:00:00 GMT

Once upon a time there was an Americans company called Motorola, and they decided to implement GSM. Unfortunately they decided to deviate significantly from the specification and implement their own proprietary back-haul protocol between BTS and BSC, called Mo-bis. It replaces the standardized A-bis interface.

Today, There are plenty of phased-out Motorola Horizon / Horizon II macro BTSs that have been phased out. Basically you can get them for scrap value, which makes them an ideal target for GSM enthusiasts willing to run a single-cell network with little investment. So while there are actually people who are interested in operating a power-consuming device roughly the size of a washing machine in their home/office - they are normally not interested in running a 19" rack sized Motorola BSC with it. Also, the BSCs are much less frequently to be found compared to the BTS.

So it would be great to support Mo-bis from within OpenBSC. A couple of brave young men have set out to try the seemingly impossible. There's absolutely zero documentation available on that protocol, and no wireshark support either. However, the University of Brno (Czech) has a functional Motorola BTS + BSC setup, and I was able to obtain protocol traces from them and actually experiment with the equipment in their lab.

The entire Motorola GSM architecture seems to be over-engineered without end. Basically you are looking at a distributed computer from the early 1990ies. Lots of processor cards (m68k, ppc) interconnected by HDLC links on top of synchronous 2Mbps links with 64k timeslots. Those links are available e.g. on the backplane of the BTS as a TDM highway. So basically even inside the BTS, the individual processors talk over E1 to each other. In the BSC, there is a token ring based LAN between some of the cards instead. And the MCUF in the BTS even supports to transport those proprietary inter-cpu links via fiber optic (!).

Each processor has a 16bit identifier by which it can be addressed in form of physical addresses. Individual processes on the processors have fixed process identifiers, and they allocate a variety of mailboxes in which they can receive messages from remote processors. There are routing functions at intermediate notes.

So any process on any processor card can send messages to any mailbox of any other process on any other processor, independent of its physical location (locally at the BTS, or at the remote BSC, or even at remote BTSs).

Besides physical addresses, there are also functional addresses. Thos addresses are used particularly to support fail-over. Every board in a BTS and BSC can be fully redundant, and if you use physical addresses, you would address one of the two redundant boards. Using functional addresses, you address the function they both can perform, and some routing magic will make sure it ends up at the current active node in the pair.

There are multiple processors in every TRX, and a couple of processors for each BTS, processors in the E1 line cards, etc. Now speaking of the actual Mo-bis interface: It seems to be a weird mixture between 08.58 (RSL) and 08.08 (BSSAP/BSSMAP). However, after staring at the messages sufficiently long, I have been able to write a more or less complete wireshark dissector for them. Radio Channel Activation (RACH/IMM.ASS) are for example handled directly inside the BTS, they don't exist as transactions on the Mo-bis like they do in A-bis.

So implementing the actual location update / MO+MT voice call and SMS related transactions is actually not all that hard. What makes things really difficult is the way the BTS is initialized at startup. Basically what resembles the OML part of standardized A-bis.

There is a lot of low-level management and bring-up of the individual processes and boards, and the download of a large 500 kByte-sized BLOB simply called database. This binary database contains literally hundreds of configuration parameters for the BTS and its neighbors. It also contains sophisticated configuration of the message routers, the switching/multiplexing of 64k timeslots on the various links, information on redundant paths within the back-haul network, etc.

Interestingly, using the password combination 3beatles and 4stooges on any of the serial consoles of the BTS or BSC, you can enter into a "god-mode" which permits you to enter the executive monitor (EMON). The executive is the operating system they run on both m68k and ppc processors. It provides access to something like a syslog of messages from the various processes, and you can manually generate messages that are to be sent to mailboxes of processes. You can inspect the object table (application programs an databases), read/write to PCMCIA flash cards, read and write to logical and physical memory, inspect CPU and I/O usage and much more. In fact, the integrated Code Object Manager (COM) even allows the processors to synchronize their code versions and remotely boot other CPUS via HDLC channels.

For a communications system geek like myself, it's extremely fascinating to see such a sophisticated and versatile system. I only wonder why on earth somebody would come up with something as complex, only to connect a couple of BTSs to a BSC. Thus, the only logical explanation is that Motorola has developed this distributed proprietary computing system way before they went into GSM, and they probably just recycled it as it already existed.

If anyone knows more about the history of this, I would be excited to hear about it. It literally feels like being an archaeologist. Analyzing ancient technology from our forefathers. But then, it only is 20 odd years old. The only time I had a similar feeling was when I briefly came in touch with IBM mainframes in 2001 and looking at IBMs SNA protocol stack.

New OsmocomBB RSSI monitor firmware

Harald Welte — Fri, 27 Jan 2012 19:00:00 GMT

Jolly has been hacking up a nice new RSSI monitoring firmware application for OsmocomBB.

I let the pictures speak for themselves:

I really hope this trend continues and we'll get some actual user interface in OsmocomBB at some point this year..

OP25 project joins hosting on osmocom.org

Harald Welte — Tue, 24 Jan 2012 19:00:00 GMT

Some days ago, I noticed that the famous OP25 project (a Free Software implementation of the APCO25 system, a digital trunked radio system) was no longer reachable on-line. It seems they were running this on a desktop PC in a university. As nobody in the project still seems to be at that university, a change in the network configuration had accidentally rendered the website unreachable.

After some quick e-mails, I offered to host them within the osmocom.org family of Free Software Projects for mobile communications. This is when op25.osmocom.org was created, and a full-site backup uploaded + installed.

I'm really happy that we were able to do a small part to help to make sure this valuable project remains accessible to interested parties in the signal processing and mobile communications field.

GSM Security Training at DeepSec

Harald Welte — Sun, 16 Oct 2011 19:00:00 GMT

Dieter Spaar and I will be holding yet another GSM security training on November 15/16 at this years DeepSec conference in Vienna.

We have been giving a series of successful GSM Security trainings in-house at various operators, as well as at a variety of conferences during the last couple of years. If you want to beef up your knowledge on the detailed inner workings of mobile networks, with a specific focus on security related aspects, this training might be a great opportunity.

You can register here. Unfortunately the Early Bird discount has already expired, but you still have a chance to book before November 2nd, after which a late booking fee will apply.

First OsmocomGMR code release

Harald Welte — Sat, 15 Oct 2011 19:00:00 GMT

The OsmocomGMR project. GMR is Geo Mobile Radio, the specifications for (among others) the Thuraya satellite telephone network.

For more details see the OsmocomGMR announcement.

I still remember some years ago, when Dieter and I were first working on some code to implement the GSM protocols, which later ended up becoming OpenBSC. A number of other developers joined the project ever since, and we have a wide user base, from individuals over academia up to commercial deployments. Next we did GPRS, which was another journey into a new world. While OsmoSGSN still has some bugs here and there, it has come a long way ever since.

In December 2010 at 27C3 I had this crazy idea of looking into yet another communications system (TETRA). Just one week of coding later, the first working code emerged and later became OsmocomTETRA. Again, history repeated itself and what was started by one person became a collaborative effort in very short time.

Finally, in July 2011, I thought it would be time for yet another communications system: ETSI GMR, used by Thuraya. This time I was too busy to actually write any code, but I just read specifications, found a supplier for some equipment and got some fellow Osmocom developers interested in it. For weeks, the IRC channel was flooded with daily reports about progress, new measurements/traces that had been made and about new code that had been written. About three months later, the code is capable of demodulating, decoding, de-interleaving, and it can give you a BCCH protocol trace in wirshark.

With this pace of progess, I wonder where we might be in yet another one or two years. At least on my personal agenda are the following items:

Finish Erlang TCAP + MAP implementation, which will allow us to implement a true HLR/AUC and finally a new MSC that can interoperate with GSM/3G core networks
Integrate OpenBSC and OpenBTS, especially now that we already have the BTS-side A-bis implementation as part of osmo-bts
Get funding to implement a GPRS/EDGE PCU, enabling osmo-bts to talk to OsmoSGSN
Work on some hardware+software interface that allows us to use the Motorola Horizon Macro BTS with OpenBSC, or at least their TRXs (called CTU) with osmo-bts
Implement a UMA/GAN gateway (for UMA capable phones and femtocells)
Support IuCS/IuPS from our MSC and SGSN for 3G Femtocells
Complete the SIMtrace firmware/software to do full MITM and SIM card emulation
Work on automated regression testing for osmo-bts, OpenBSC, OsmoSGSN and all other GSM related Osmocom components.
Continue the excellent work that has been done on supporting MTK chipsets from OsmocomBB at some point

At least now you know there is never any reason to be bored. If you have time and are interested in helping with implementing any of this stuff, let me know.

Ground-breaking research on APCO P25 security

Harald Welte — Sat, 17 Sep 2011 19:00:00 GMT

While we at OsmocomTETRA have been looking only at implementing the TETRA protocols as they are (and doing a bit of sniffing on unencrypted networks), some researchers have recently published two ground-breaking papers on the (lack of) security in the APCO P25 radio system.

In case you haven't heard about APCO P25: It is a digital mobile radio system mainly used by Police in non-EU English speaking countries like the US, Australia and New Zealand.

You can find the respective papers here and here.

So apparently P25 uses either single-DES or a proprietary cipher with only 40 bit key-length. No, I'm not joking. Seems like it was developed by people who have not the slightest clue about communications security at all.

And guess what they used to receive and transmit P25 waveforms? Of course the USRP and gnuradio. This once again proves how invaluable those tools are, not just for the FOSS community, but also for the communications research community.

Major bugfix release of SIMtrace firmware

Harald Welte — Mon, 15 Aug 2011 19:00:00 GMT

At the CCC Camp 2011, the Osmocom SIMtrace project was a major success. Not only were something like 60 units out of our initial batch of 100 units sold, but the SIMtrace workshop was so successful that it had to be held three times instead of once.

During the workshop we discovered a very annoying bug which I wasn't able to solve immediately. Depending on the combination of phone/simcard used, the SIMtrace would disconnect from USB and the phone would claim there is no SIM card inserted.

The debugging went like this:

SIMtrace was resetting very early in/after the ATR
the reset reason was diagnosed as being a watchdog reset
the watchdog was triggered by an IRQ storm from the USART
the IRQ storm was caused by the firmware not clearing some parity error / overrun related bits

However, at that point I couldn't further find the cause of the bug. I assumed it was related to the PPS/PTS, but couldn't really point my finger at it. If we sniff the PPS/PTS wrong, then of course our baud rate is different from the real baud rate, which in turn would cause perceived parity errors and the like.

I'm grateful that most people still didn't loose their interest in simtrace and happily bought the unit and/or attended the workshop.

After a bit more debugging after the camp, I have now solved the bug. I simply never realized that the TCK (ATR checksum byte) is only present in cards that support T=1 as well as T=0. However, some simpler SIM cards like the ones that we issued for our test GSM network on the camp only do T=0 and thus don't transmit TCK.

The old code thus considered the first PTS/PPS byte (0xff) as the TCK, and didn't recognize the PTS/PPS correctly.

Firmware version v0.2 fixes this problem. I've released the firmware update, now also available from the wiki

Update on the GSM network at the CCC Camp 2011

Harald Welte — Thu, 21 Jul 2011 19:00:00 GMT

During the past weeks, I've been trying very hard to get to a technical solution for the setup regarding the private GSM network that we intend to operate at the CCC Camp 2011. Unfortunately, despite puting in way too much time that I don't have, no really good solution appeared. There were times when I was wondering if it would happen at all - mainly due to the lack of properly integrated / tested RF related issues like PA, LNA, duplexer, combiner, etc.

But it seems just in time Dieter came to the rescue. So now we have pretty much figured out the equipment and settled on a configuration. We'll have 2 Nokia Metrosite BTS with a total of 5 TRX, each running at 5W using borrowed equipment.

During the next 10 days, all the parts like antennas, cabling, plugs, adapters and the BTS units themselves should arrive at my place. Let's hope there are no serious fuck-ups that cause something to not arrive in time.

So all in all, there's a 99% chance we will have a functional GSM network. The Nokia A-bis support in OpenBSC will be brand new, so there might be some glitches here and there. But then, that's part of the fun. I'm already very curious to see what kind of coverage we get. I guess if we do things right, it should reach well into Finowfurt itself, and not just barely cover the camp grounds like we had at HAR 2009.

On the recent THC release on the Vodafone femtocell

Harald Welte — Wed, 13 Jul 2011 19:00:00 GMT

I am mainly posting this to prevent any more people mailing me about this release. There's nothing really spectacular here.

Starting from 2009 on, the usual suspects (aka OpenBSC developers) have been looking at various 3G femtocells, including the Vodafone one (I have 10 of them here). Aside from that Alcatel-Lucent design that Vodafone uses, we've also looked at the Cisco/AT+T/ip.access design, as well as the Ubiquisys/SFR one. With some effort you can root all of them, and you can then make sure they don't connect to the respective operator but to an IP address of your choosing.

The protocols are vendor-dependent. The Vodafone femtocell uses a version of RANAP (the protocol between RNC and MSC in UMTS) behind an 8 byte proprietary header. As RANAP is specified in the 3GPP, it was pretty easy to build a small piece of code that interacts with the unit.

Ubiquisys (used by SFR) uses the UMA protocols, and the Cisco/ip.access/AT+T design uses a proprietary ip.access protocol called URSL (sort-of a "progression" of the 2G RSL to UMTS).

Supporting them from OpenBSC is not easy. While the call control and SMS transfer protocols of 3G are identical to GSM, everything below doesn't really bear much resemblance. I would guess it would take at least a man-month to get basic signalling, call + SMS support working, if not more.

Given the fact that the femtocells all speak their vendor-proprietary dialects, and given that they often come with license terms that only permit the use of their firmware in combination with their gateway located at the operator network, we never thought it is a high priority item for us to work on.

What you also have to consider, is that their output power of 20dBm is even less than the 200mW of typical small-scale GSM BTS, and that they typically only support the operation of 4 concurrent phones. Nothing that you would be able to run e.g. a conference telephony network on.

Furthermore, due to the wide channels (5MHz), it is very likely that all available sprectrum has been auctioned off/licensed to commercial operators, so it's almost impossible to get something like a test license. In GSM with 200kHz channels, there's often still a guard band or some unallocated channel that can be used.

If you really want to have some free software + femtocell based 3G network, go ahead and do it. The option existed for years now, ever since femtocells started shipping to the market. All of them are some form of embedded Linux systems. Rooting them isn't really different from rooting a Linux based WiFi router / DSL modem. So what's that new about the THC release? That a vendor of Linux embedded devices chose a trivial password? If you're surprised by that, I guess you haven't taken apart many embedded devices then.

SIMtrace v1.0 prototypes are working out of the box

Harald Welte — Fri, 01 Jul 2011 19:00:00 GMT

After the debacle with various wrong footprints in the v0.9, I'm very happy to announce that the SIMtrace v1.0 hardware is working fine. All footprints correct, schematics correct, layout/Gerber correct. All we had to do is solder the components, attach it to USB, flash the firmware and use it.

Here's a picture of the board (sorry, my soldering is not very clean):

Early next week we will be ordering a batch of 100 units from the SMT house we have chosen.

First working prototypes of Osmocom SIMtrace design

Harald Welte — Tue, 21 Jun 2011 19:00:00 GMT

Last winter I was working on some hardware and software that can be used to trace the communication between a SIM card and a phone and called it Osmocom SIMtrace. At that time, I was simply recycling an old OLIMEX development board for the AT91SAM7S micro-controller.

But since the firmware for the micro-controller, the host software as well as the wireshark plug-in has been written now, it would be a shame if I was they only user of the project. Therefore, Kevin Redon and I have spent some time in polishing and improving the design, as well as generate some actual prototypes.

Unfortunately a number of mistakes were made (both on the design side but also wrong component pin-outs) so there was a need for significant re-working.

Nonetheless, we now have some 5 functional prototypes, a picture can be seen in the Osmocom Wiki, where you can also find the schematics

We're now having a second version of the PCB built, this time hopefully with correct footprints for all parts. Once that is verified at the end of next week, we will give "go" for the production of a small batch (100 units).

Interested developers will be able to obtain the resulting hardware from mid-August onwards. We also expect to be offering them at the Radio Village of the 2011 CCC Camp.

Tracing the SIM<->Phone protocol can be useful in a variety of cases:

Observing the behavior of operator-issued SIM cards in terms of which SIM Application Toolkit or Proactive SIM features they use.
Debugging aid while developing and interoperability testing of your own SIM toolkit applets
Prototyping and development of SAT blocker or other SIM card firewalls which restrict the security or privacy threats originating from untrusted operator SIMs or potentially compromised SIM cards.

Exploring the Motorola Horizon macro BTS

Harald Welte — Fri, 17 Jun 2011 19:00:00 GMT

Some days ago, my new 100kg toys have arrived: The Motorola horizonmacro indoor cabinets, populated with 3 GSM 1800 TRX each. Pictures are at the openbsc.osmocom.org wiki

It took some time to manufacture the power cable, and specifically the E1 cable (where I had to reverse engineer the pin-out of a 37pin sub-d connector that the so-called BIB (balanced interface boards) use.

The next biggest time consumer was the fact that the command line based user interface (MMI) has three modes; MMI-ROM, MMI-RAM and emon. Figuring out which commands to use to switch modes isn't really something that you can easily find. Especially the fact that the MMI-ROM to MMI-RAM switching command has a parameter that needs to be identical with one stored on the PCMCIA flash card (number "18" in my case), didn't make things any easier.

So as an intermediate summary, I can make the following comments about the Motorola BTS and specifically A-bis architecture:

Motorola seems more proprietary and less specification oriented than what I've seen so far (Ericsson, ip.access, Siemens, Nokia).
They do not seem to implement a SAPI=62 OML link on A-bis at all
Thus, there is no GSM TS 12.21 compatible OML protocol at all
Instead of using individual OML messages and/or attributes to set things like ARFCN, BSIC and the like, the Motorola BSC seems to generate one big database blob containing all parameters. This blob is downloaded into the BTS RAM (optionally its PCMCIA Series2 flash card).

Particularly the latter part is causing quite some problems for me. As I don't have a Motorola BSC, I cannot generate those database files. My BTS units come with databases on their PCMCIA flash cards. I can view their contents on the MMI. However, their config (EGSM) doesn't match the actual radio hardware that's installed. Even after hours spent with the MMI, there seems absolutely no way how those parameters can be altered locally

I also have not found any hint / documentation at all about something like a LMT (local maintenance terminal) like other BTS vendor. Using such a software on a PC, you can typically configure the BTS via a RS232 line.

So most of my hope now lies in being able to analyze dumps of those old Series2 flash cards in order to get some hints on that database format.

If anyone has any of the following information, it would make my day:

Motorola A-bis / Mo-bis protocol traces
Any Motorola BTS config databases (independent of BTS model/version)
The sample database files that come with a Racal 6113 Option 225
Any information on the database format

But to be honest, I don't have much hope. The equipment is old (about 1999), and only very few operators have been using it, as it seems.

ETSI and its ridiculous fees for old archived documents

Harald Welte — Mon, 06 Jun 2011 19:00:00 GMT

I am currently looking for some old meeting minutes in order to understand who was the driving force behind certain features in GSM.

Ever since the GSM standardization had been handed over to 3GPP, all meeting minutes are freely accessible and downloadable for everyone. But what about the 15-20 years before that? They remain in the ETSI archive.

So from April 2011, the ETSI has started to offer an archive DVD, containing all the early CEPT and ETSI documents such as draft standards and meeting minutes. What a great idea. This DVD set is titled A Technical History of GSM Standards

But then, when you look at the price tag, you can only think "Seriously? They must be kidding!!". They are selling it for 6,000 EUR. Yes, this is not 60 EUR, not 600 but 6,000!. Go and see with your own eyes at the ETSI web-shop or this flyer.

But if that hefty price was not enough, they add an additional burden: You have to be an ETSI member to even buy it. And what is the cheapest option? Well, as an individual/small business you can join for a reduced price of EUR 3,000 per year. So in order to get access to some old meeting minutes from the 1980ies or 1990ies, I have to pay a total of EUR 9,000? They must be out of their freaking minds. Sorry, but I am simply lacking any other words how I could put it.

I think ETSI and the entire telecomms industry can be happy if anyone shows an archaeological interest into ancient specification texts at all. Scaring them away with a more than ridiculous price tag is certainly not going to encourage students or researchers to understand who, how and why GSM has ended up what it is today.

Looking for documentation and/or protocol traces for Motorola Horizon BTS

Harald Welte — Tue, 31 May 2011 19:00:00 GMT

It seems like I'll be getting my hands on some Motorola Horizon 1 BTS soon. Of course it would be great to add OpenBSC support for yet another vendor / model.

So if anyone out there has any information on Motorola Horizon, I would be more than happy. Information includes:

Motorola A-bis (Mo-bis) protocol traces
Motorola A-bis (Mo-bis) protocol specs
Installation manuals
Configuration manuals
Service manuals

Thanks in advance!

Finally: A parseable MAPv1 ASN.1 specification

Harald Welte — Mon, 11 Apr 2011 19:00:00 GMT

After way too much work in extracting the ASN.1 text from word documents, learning about the differences in pre-1990 ASN.1 and pre-1990 TCAP with their respective current-day counterpart, some futile attempts with the incomplete ASN.1 grammar files available for ANTLR3, I have finally dedicated the entire day to manually convert the MAPv1 from ETSI TS 09.02 3.10.0 (1994) to work with present-day TCAP and present-day ASN.1 syntax.

The result not only passes the syntax and semantic checker, but it is accepted by the Erlang asn1ct. It has not been tested/validated against any test data so far. Just in case anyone else needs a 'working' version of MAPv1 some 20 years after it was created, I have published the result here: http://cgit.osmocom.org/cgit/asn1_docextract/tree/output/3.10.0

Any bugfixes/improvements/complaints are obviously welcome.

Now I just need to figure out how to properly work with the ROS/ROSE CONTRACT class and TCAP APPLICATION-CONTEXT class to formally describe the various application contexts, their respective MAP operations and associated application context versions. It's actually quite a bit surprising that the ETSI/3GPP don't specify this formally in 29.002. The information is contained in the human-readable part of the document, but not in the formal ASN.1 notation. I wonder why...

Following up on my GSM MAP archeology project

Harald Welte — Fri, 08 Apr 2011 19:00:00 GMT

In the past weeks, I have been blogging about the work I've been doing in trying to recover the earlier versions of the GSM MAP ASN.1 files in order to support it from my own Osmocom Erlang MAP code.

After having gone through the tiresome exercise of implementing a custom MS Word for DOS file parser to extract the ASN.1 definitions from the 3GPP-published DOC files, I tried to feed them into the Erlang asn1ct code generator.

Unfortunately this didn't work, as the old MAP ASN.1 references very old ITU-T TCAP from 1988. So I went to the ITU website. There again a similar story. From their asn1 website section, you can only get the TCAPMessages from 1997. If you want an older format, like the Q.773 from 1988, you can download a PDF file. Most of that PDF file is actual text. But the ASN.1 sections are included as an image, so there is no way how you can copy+paste them.

So there I went, and manually typed in those few pages of ASN.1 source code. After I was finished, I thought I had finally done enough. Writing a word-for-DOS parser, typing in pages of ASN.1 manually. What more can there be? I should be able to finally generate Erlang code for decoding and encoding the respective messages.

But it would be too simple if it was that easy. The 1988 TCAP as well as the old MAP specifications use the ASN.1 "MACRO" construct to define the individual MAP operations. However, as it seems the MACRO construct had been deprecated in 1994 by the ISO in charge of ASN.1. Successive ISO specs for ASN.1 have completely remove the MACRO construct as similar results can be achieved with newly-introduced information objects. So of course, the Erlang asn1ct (as well as other free software ASN.1 tools) does not support such an antique feature.

So here I am in the year 2011, unable to use information available in the public ITU-T and ETSI/3GPP specifications to build a GSM MAP protocol implementation that can deal with the messages that you encounter on real-world SS7 links. I guess there are now the following different approaches to proceed:

write a converter that translates MACRO into OBJECT
manually convert the old MAP specs from MACRO to OBJECT
add MACRO support to current-day ASN.1 tools
find some old ASN.1 code generators that still support MACRO

All in all, this is a really dissatisfying situation, and I would say it is a failure of the standardization bodies to provide useful versions of their specifications. You cannot remove operations and associated message types from a specification, while in real life even 15 years later a whole number of users still use messages according to those (now removed) definitions. Meanwhile, the old specs are getting useless, because you not only deprecate but completely remove a language construct of the formal specification language used.

This renders the idea of having a formal specification useless. If I first need to write my own converters or manually convert that very same formal specification, errors can be introduced in the translation/conversion process, just as much as errors could have been introduced in manually writing encoding and decoding routines based on a non-formal-specified protocol like most of the IETF / Internet protocols.

More MS Word for DOS file parsing in the name of GSM

Harald Welte — Sat, 26 Mar 2011 19:00:00 GMT

It seems that from TS 09.02 version 4.2.0 on, the editors have actually put markers as "hidden text" inside the Word document, which allow better automatic detection when a given ASN.1 module starts, is interrupted by plain text, continues and ends. The following screen shot (from Section 14 of the above-mentioned document) is human-readable hidden text explaining the syntax:

So now I'm adding this format as a second option to my extraction tool.

Please note: The tool I wrote yesterday (working fine with version 3.x.y of 09.02) is available from asn1_docextract.git. Later today it should also support the >= 4.2.0 annotations outlined in this blog post.

Implementing a custom MS Word for DOS file parser to properly do GSM SS7

Harald Welte — Fri, 25 Mar 2011 19:00:00 GMT

Yes, I'm not kidding!

In recent months, I've been writing quite a bit of GSM MAP (Mobile Application Part) code. MAP is the protocol used heavily in the GSM core network and especially on the roaming interfaces between different operators. It is specified in GSM TS 09.02 and later 3GPP TS 29.002.

The protocol specification relies on ASN.1 description of the messages as well as the regular BER encoding rules. ASN.1 is this marvelous technology that allows a protocol to be specified in an abstract and formal notation, in an extensible way, removing all the problems of human-written marshalling code, full of errors and differences due to different developers interpreting a human-readable specification in different ways.

So far so good. You think it should be simple to write a parser and generator for MAP messages: Simply feed them into the ASN.1 compiler of your choice, it will generate code in the target language you require.

As long as both sides of the communication do that using exactly the same revision of the specification (and don't make implementation mistakes), this will work. The reality looks very different, though :( When I test my code against something like one million of real-world messages captured on a production SS7 roaming interface, it produces errors already on packet number six of that trace.

The problem is: The protocol designers have not specified the first versions in a really extensible way, i.e. a given operation originally only returned one atomic data field, and it was later extended to return a sequence of data fields. Thus, there is one additional level of hierarchy in the encoding.

Not only that, but in their infinite wisdom, the designers of MAP have also failed to include versioning information in each and every message header. Instead, it is part of the application context name, which is only part of the first message of every conversation.

Furthermore, different versions of the MAP specifications disagree on whether certain fields are deemed optional or not. This is further complicated by somewhat strange versioning habits. There is the Revision number of the TS 09.02 (like 3.8.10), then there is a different version number encoded in the corresponding ASN.1 files like 'version9(9)' and individual operations then have v1/v2/v3 in their application context name.

Some even more wiser decision must have been to remove the description of older messages from the later versions of the specifications. So even specifications published in the year 2000 no longer include definitions of messages that were still part 5 years earlier. Why does it matter? Because today, in 2011, you still see MAP message on the international SS7 interfaces that are encoded in some of the earliest versions of the MAP protocol!

And if all of this was not enough, the biggest bummer is: For most of the releases of the specification, the ASM.1 text files are not distributed separately, but they are interspersed with human-readable text in the actual specification documents (which can be 600 pages long, nothing you want to cut+paste).

Even worse: If you go to the ETSI homepage and download the PDF version of old 09.02 specs, they will actually provide a PDF with a scanned paper print-out, i.e. no searching and no copy+pasting.

Luckily, the 3GPP has made the history of 3.8.0 and later available on their FTP server. But they are in MS Word for DOS format, like they were written originally. This format can not be opened by OpenOffice, and as far as I know not even by any of the Windows Word versions that MS has released in the last 10 years.

So what did I do? I actually installed MS Word 5.5 for DOS (provided as Freeware from Microsoft) and ran it in DOSEMU, to convert the specs into RTF format. This way I can at least open them and look at them in a modern text processor.

But this still does not solve the copy+paste problem.

I finally found antiword, but it mainly focuses on Word for Windows files and only does rudimentary text extraction from Word for DOS files. But hey, there is an online copy of chapter 16 from the File Formats Handbook, apparently published by Dr.Dobb's (who remembers them!!) at some time in the past.

So what did I do? I wrote some custom parser for those old Word/DOS files, which parses the paragraph format descriptions and tries to identify those sections that contain the ASN.1 code. As they are almost the only part in the specification that is enclosed with a border line on all four pages, this should work pretty fine. Early results are quite promising!

My hope is now that the ETSI stylesheets did not change too much over time, i.e. that this parser will be able to extract the ASN.1 spec for all of the protocol versions that I can find. If that works, I can run them through a validator, then pretty-print them and putt them all in one git tree in chronological order. And maybe at some point in 2011, we will have the marvels of an unified diff between two different MAP versions. The strange part is: Diff was developed in the 1970ies, GSM in the late 1980ies. They should have known about it back then, and used a revision control system like SCCS to record all the changes in the specification they make.

I guess this all is a glimpse how a digital archaeologist of the 22nd century must feel when analyzing ancient artefacts and trying to understand what the heck his ancestors have been doing back then.

UPDATE: The tool can be found at http://cgit.osmocom.org/cgit/asn1_docextract/

Starting to experiment with Anritsu MD8470A signalling generator

Harald Welte — Thu, 17 Feb 2011 19:00:00 GMT

Earlier this week I was able to pick up some new equipment (aka toys) from the customs at Berlin TXL airport. Among other things I learned that despite filing an Internet-Zoll-Anmeldung (IZA) (Internet Customs Declaration), online via the German customs web site, you still have to print two copies of it on actual paper. I only had one copy, and the customs department does not have a copier to produce another copy. Buerocrats :/

In any case, I am now the proud owner of an Anritsu MD8470A signalling generator, which is basically a small 3G (WCDMA) network on steroids, all inside a single box. It can do mobility management, call control, voice calls, sms, packet data service, WAP, etc. It even has a legacy GSM/GPRS radio, so you can do inter-RAT hand-over between 3G and GSM.

But what is even more exciting: It includes (proprietary) APIs that allow you to send sequences hand-crafted messages on RRC and any layer above. This means it is an excellent tool for security and robustness testing of mobile phones.

Struggling with adding Ericsson RBS support to OpenBSC

Harald Welte — Sat, 12 Feb 2011 19:00:00 GMT

I've been spending way too much time recently in understanding the low-level aspects of Ericsson RBS 2000 and the associated OM2000 protocol. The goal here is to support this family of BTS from OpenBSC.

The first big obstacle was that the A-bis Layer2 (LAPD) is quite different from what we've seen with Siemens BTS before - and also from what the GSM Specs TS 08.56 says.

In the Ericsson A-bis, there are the following key differences regarding standard A-bis:

E1 timeslot is not configured statically. Instead, the BTS scans the entire E1 link and looks for SABM messages to TEI=62/SAPI=62
There is no TEI manager
LAPD sessions are initiated from BSC to BTS
There is not only one OML connection for each BTS, but one additional OML connection for each TRX
OML does not follow 08.59/12.21 but is proprietary

All those parts above have now been solved. We can initialize the A-bis link and talk OM2000 to the DXU/IXU of Ericsson RBS 2000, and we can use that to configure and initialize the CF (central function), as well as to configure the IS (Interface Switch) and CON (concentrator).

However, the IS configuration is already quite difficult. In that configuration you connect 1:1 mappings of various ICPs (Interface Connection Points). So you can connect any 64k or even 16k sub-slot of a TRX with any of the E1 interface(s) timeslots. However, the assignment of which TRX(TRU) is represented by which ICP is BTS specific. On a RBS 2401 for example, the first TRX (TRX0) is attached to ICP 512..523 (1x 64kbps signalling + 8x16kbps traffic).

So if we configure the IS to connect those ICPs 512..523 to the ICPs 4..15, then we will get the TRX0 routed to Timeslot 1,2 and 3 of the first E1/T1 interface. You can see some examples in page 89 (pdf page 115) of this document. This works on the RBS 2401, but it seems like the RBS 2308 has different ICP/DCP assignments than the generic RBS 2000 example that they are showing.

If anyone is more familiar with the details of the Interface Switch in RBS2000, and specifically the ICP / DCP mapping inside the RBS 2308, I would definitely want to have a chat with you.

If we cannot figure this out, it is impossible to bring up the per-TRX OML and RSL links, and thus impossible to use those BTS with OpenBSC :(

SS7 work: M2UA, MTP3 and ISUP message {de,en}coding

Harald Welte — Fri, 14 Jan 2011 19:00:00 GMT

One of my paid contracts required me to start moving directly into the GSM core network, the SS7 domain. While so far I had a fairly good understanding of SCCP, TCAP and MAP (which are required for GSM/3G roaming interfaces), I've now had to look at the actual telephony signalling side of things.

Even in GSM networks, the gateway or inter-working MSC will translate all the GSM TS 04.08 Call Control messages into ISUP, the ISDN User Part, originally designed for call control signalling between ISDN networks.

As apparently always in SS7, there are many, many different standardized and proprietary protocol stackings that can be seen in the wild. I'm now working on a stack that looks like IP->SCTP->M2UA->MTP3->{ISUP, SCCP->TCAP->MAP}.

Luckily, for now there is no need to do any fully-fledged implementation of it, simple message parsing and encoding routines are sufficient for the task at hand.

It's been about time that I'm closing those last gaps in the knowledge of GSM core network protocols. The only part that I'm still missing so far is CAMEL. I know roughly how it works, but I've never played with it or implemented any part of it.

Supporting the HSL 2.75G femtocell from OpenBSC

Harald Welte — Thu, 13 Jan 2011 19:00:00 GMT

The last couple of days I've been hacking away on reverse-engineering the proprietary Abis-over-IP variant of the HSL Femtocell. This is required to get this latest newcomer in the GSM femto/picocell market to work with our OpenBSC (and later OsmoSGSN) software. Progress is quite good now, apart from their custom RTP multiplexing format, everything required for signalling, SMS and Voice is working from OpenBSC.

The HSL Femto is a nice and powerful piece of hardware, containing a TI DaVinci ARM+DSP chip, 128MByte DDR2 memory, a Spartan-3A FPGA and 275Ms/s DAC as well as 65 Ms/s ADC. Much too powerful for a single-ARFCN GSM system. This really looks like the vendor wants to do multi-ARFCN software updates later. More details and some initial PCB photographs can be found in the OpenBSC wiki.

The Software side looks a bit like it is still maturing. Most bugs I have found so far are apparently fixed in the SR1.0.1 firmware. The A-bis dialect is quite different (and more simplistic) from what I've seen in any other BTS. More details can once again be found at a page in our wiki.

What's exciting is that there now is a commercially available traditional BTS product at relatively low cost. By traditional I mean it is still only a BTS and not a Um-SIP gateway like OpenBTS.

I hope this will enable more people to use and experiment with OpenBSC, as the cost and availability of the ip.access nanoBTS has always been an issue for many people without a four-digit budget available.

OpenBSC field test at 27c3 over

Harald Welte — Thu, 30 Dec 2010 19:00:00 GMT

During the last week I was busy with

December 22nd though 24th: Preparing OpenBSC to be ready for the field test at 27c3, i.e.
- improving the output of the log at "INFO" level to be not too verbose at the expected network load
- Implement the interface to LCR using a Unix domain socket rather than linking LCR with OpenBSC
- Configuring all 6 BTS, put them in multi-TRX config, test the setup
- Manufacturing nanoBTS stacking cable (with their weird RJ-69 plugs that you have to mill a notch off)
- Install all required software on the machine that will run OpenBSC
December 25th and 26th: Setting up the network
- Physically mounting the nanoBTS units
- Patching cables throughout the building, installing PoE switches
- Configuring LCR
- Interfacing with the Phone Operation Center (POC) via E1 / DSS1
December 26th through 30th: Running the network
- Fixing bugs as they appear (see e.g. zeckes mailing list post
- Making sure users are happy
December 30th: De-installing the network

I don't have much time now, still have to unpack lots of boxes full of gear. However, I have finally completed my scripts to graph some of the statistical data of the field test. You can see the graphs in the OpenBSC wiki.

Unfortunately we don't have the same body of statistical data for the previous field tests at 25c3, 26c3 and har2009. However, for all of those three events we have now graphs about the IMSI/Country distribution of all the phones that have ever tried a LOCATION UPDATE with us: 25c3, 26c3, HAR2009, providing some nice statistics on what nationalities are attending the respective events.

Interview about GSM security (in German)

Harald Welte — Mon, 20 Dec 2010 19:00:00 GMT

The major Austrian newspaper Der Standard was yesterday featuring an an Interview with me on GSM security related issues. Being in Austria, the interview is obviously in German language, sorry for all non-German-speaking readers of my blog.

Preparations for GSM network at 27C3 conference

Harald Welte — Fri, 17 Dec 2010 19:00:00 GMT

Behind the scenes, we've been working on preparing the experimental GSM/GPRS/EDGE network at the 27th Chaos Communication Congress. The regulatory authority was nice enough to grant us 6 ARFCN, which we will split to 3 BTS (2 TRX each), resulting in one BTS with 2 TRX on each of the 3 conference building floors.

I've started a page in the 27C3 conference wiki about the GSM network. Please notice that this information is still preliminary at this point.

The wiki page also contains detailed instructions on how you can participate in the test network. I'm hoping a lot of you will bring a dedicated cellphone that you can put the 27C3 SIM inside and participate in the network.

I'm particularly excited about GPRS/EDGE support. We will be handing out official, world-wide routed, unfiltered IPv4 addresses to each and every phone. This means you are free to run port scans or other attacks (please: No DoS) over an unfiltered IP network directly into your mobile phone.

Learning how GPS _really_ works in order to truly understand RRLP

Harald Welte — Thu, 16 Dec 2010 19:00:00 GMT

Back one or two years ago, when I first discovered the RRLP as a mechanism how operators can get very precise GPS positioning of a mobile phone (without any authentication or a way for the user to prevent or at least notice it), I was frankly speaking shocked.

We've done some experiments at HAR2009 and obtained a number of great position fixes, mostly from iPhones. The nice aspect of RRLP is that it is buried down inside Layer3 of the GSM protocol stack in the baseband processor. This is at a much lower level than all the web or App based location based services that are running in an application program in userspace of the application processor.

Now RRLP comes in a number of different flavors. What we have done so far is called ms-based positioning, where the phone works as an autonomous GPS receiver, pretty much like a personal navigation device or any hand-held GPS receiver. So the network simply asks "tell me your GPS coordinates if you know them" and the phone will respond. Some phones ask for assistance data in order to do A-GPS. But that's it.

What has been more of a mystery to me is the ms-assisted GPS RRLP mode, where the phone just performs some measurements and forwards the resulting data to the network. I never really understood the details of how it works, but always wanted to. Last week I finally found some time to do the research required to fully understand it:

The network tells the phone the exact bit timing, Doppler shift and other parameters for each of the satellites that it _knows_ the phone would be receiving given the current cell the phone is registered to. The phone then performs some measurements within very narrow time/frequency/synchronization windows, and passes back the timing of those received signals relative to the current GSM cell signal. Using this information, the actual position estimate will be completely computed inside the network, not inside the phone.

Presumably this ms-assisted mode was implemented to not have to put a full-blown GPS receiver into every phone, requiring sophisticated processing in either hardware or software. Also, this method should be much quicker as the network _knows_ all the current ephemeris data and GPS signal timing, whereas a stand-alone GPS receiver would have to take quite a lot of effort to acquire a signal from cold-start, even if there is some assistance data.

Unfortunately I don't have the time to actually implement the network side for this. It would be a fun project, but I have already way too many of them (and customers who only pay for other features in our Free Software GSM stack).

There's now a RRLP wiki page at security.osmocom.org. As short as it is it still contains more information about RRLP than I could find on any other public source on the network - except the protocol specs.

Wireshark patches enhancing IPA Abis/IP dissector accepted

Harald Welte — Thu, 16 Dec 2010 19:00:00 GMT

The wireshark project recently accepted two of my patches related to the Abis/IP dissector. The first of them makes the TCP/UDP port numbers that are interpreted as IPA multiplex a configurable preference. This is really useful, as the actual port numbers used in production setups seem to differ from site to site (with no real standard port numbers and only some that are 'best practise'). Without this patch, in many case you always need to click 'Decode as... GSM IPA' every time you open a pcap file.

The second patch adds support for printing the debug messages that the Hay Systems Ltd. HSL Femtocell includes as stream identifier 0xDD in its Abis/IP variant.

I hope I can find some time to clean up / finish some of the other wireshark patches that we have pending for quite some time. The main problem here is that we imported some definitions from OpenBSC, which use gcc extensions and are thus not permissible for wireshark inclusion.

Starting to work on drivers for the Mediatek MT6139/MT6140 RF Transceiver

Harald Welte — Sat, 11 Dec 2010 19:00:00 GMT

In the last two days, I have finally started to get some initial work done on the OsmocomBB port for the Mediatek chipsets. My current focus is the MT6139/6140 RF Transceiver, including stuff like setting it up for Rx and Tx, computing the VCO/PLL dividers for all ARFCNs in 4 bands for uplink and downlink, etc.

Next will probably be the drivers for the MTK digital baseband BPI (baseband parallel interface) and BSI (baseband serial interface), which are needed to actually use the MT6139/6140 driver, as well as the antenna switch module, power amplifier and other parts of the RF front-end.

I'm not really testing any of this code at the moment, as I'm travelling a lot without access to my Racal 6103 or other GSM handset testing equipment. However, writing even untested code helps me understand the chipset better and is a first step in the right direction. I guess January 2011 will provide more time to continue/complete this task.

Trying to understand Ericsson Abis-IP + OML

Harald Welte — Sun, 28 Nov 2010 19:00:00 GMT

I've recently been able to look at packet traces of an Ericsson BTS (which they call RBS) and have been working on understanding their proprietary Abis-over-IP protocol stacking and OML layer.

By now, all equipment makers have migrated their BTS products from classic TDM (E1/T1) lines to some form of IP-based back-haul. However, the GSM specs as published by ETSI and 3GPP still only specify the E1/T1 transport layer, and every vendor seems to invent their own protocol for back-haul.

What we already know (and support) is the ip.access style Abis-over-IP, where they have their IPA multiplex layer inside TCP. Inside that they then have pretty straight-forward 08.58 (RSL) and 12.21 (OML) messages.

With Ericsson, they use a stacking like this: Ethernet, IPv4, L2TP, custom-HDLC, RSL/custom-OML.

The custom HDLC layer (I have called it Ericsson HDLC, or short EHDLC) seems to work quite different from all other forms of HDLC that I've seen:

1 Byte Address
1 Byte Length (including the header!)
1-2 Bytes Control
n Bytes Data

The Control octets are just like in any HDLC, i.e. you have U/I/UI/S Frames and see commands like SABME, UA, RR, ... It mostly uses a two-octet control word that has both N(R) and N(S) for acknowledgements. At the beginning they actually do a XID exchange that seems compliant with ISO 8885 (if you cannot find that document, try reading the AX.25 spec instead, its XID is compatible with ISO 8885).

I have not fully understood the Address octet yet. I see lots of changes in the upper three bits, and it seems there is a SAPI or TEI that is the lower 5 bits of the octet.

Having a length field in an HDLC header instead of any flag bytes is indeed very uncommon to see.

The OML layer (called OM2000) is completely proprietary and shares nothing with the GSM specs 08.59/12.21 apart from the three byte header. However, I have managed to build a pretty complete dissector which you can find together with the EHDLC code in this rbs2409 patch which applies on top of the generic wireshark abis_oml.patch

It is my hope that this information (and particularly the dissector) will prove a valuable resource once we add Ericsson BTS support to OpenBSC. If there is anyone who can provide us real BTS (RBS) hardware, please let me know :)

Announcing Osmocom SIMtrace: A smart card sniffer

Harald Welte — Thu, 18 Nov 2010 19:00:00 GMT

During recent weeks I started to do some work related to SIM Application Toolkit (STK / SAT). Debugging this kind of application is hard, as you never really know what exactly is going on between your SIM and the phone, and you don't have the full source code for either of them.

Thus, the need for passively sniffing/tracing the smart card interface between SIM and phone was born. There are commercial solutions which are not only prohibitively expensive, but then they are again proprietary/closed, i.e. you cannot extend them how you want.

There are also some free/open projects like the good old Season scanner, or the slightly more modern RebelSIM Scanner. However, those are really dumb and you have to manually determine the bit-rate using an oscilloscope and then program the UART accordingly. Furthermore, their top speed is often limited. None of this is really useful if you e.g. want to test a variety of combinations between N SIM and M phones, where you don't want N*M times of manual determination of bit-timing on an oscilloscope.

As an alternative solution, I have now created Osmocom SIMtrace. It uses an AT91SAM7S micro-controller as hardware interface between the SIM card interface and USB. It properly sniffs the RST, CLK and I/O lines of the SIM and does auto-bauding, follows negotiation of new bit-rate negotiation via PPS/PTS and re-assembles / segments the APDUs as they come by.

Finally, the APDUs are picked up by a small command-line program that feeds them into wireshark, where you can inspect them like any other communication protocol that you're used to.

The code is still fairly experimental, but for anyone with an interest in this topic it should definitely be possible to reproduce my results.

There is not much specific to SIM cards in this project, by the way. It should work with any ISO 7816-3 T=0 smart card. Adding T=1 is just a matter of software, if you need that protocol..

And now I'm finally off to doing the actual work that I wanted to do.

Initial mt6235 Linux and u-boot code available

Harald Welte — Thu, 18 Nov 2010 19:00:00 GMT

As Marcin announced yesterday on the OsmocomBB mailing list, his initial u-boot and Linux port to the MT6235 baseband processor has been pushed to the git.osmocom.org server. He has also provided some instructions and pre-compiled kernel and u-boot images.

He's now working on the NAND, SD/MMC, GPIO and LCD drivers. If you want to help out, feel free to contact Marcin about this.

Meanwhile, I've been doing a bit of theoretical analysis on the GSM baseband / RF interface of the MT6235, based on the limited documentation that is available to the general public. Seems like it's about time to start with practical experiments soon..

A brief history on the withdrawal of the A5/2 ciphering algorithm in GSM

Harald Welte — Thu, 11 Nov 2010 19:00:00 GMT

Recently, I wanted to investigate when and how A5/2 has been withdrawn from both GSM networks and GSM phones alike. Unfortunately there was no existing article discussing this history online, so I went through dozens of meeting reports and other documents that I could find online to recover what had happened.

If you don't know what this is all about: It is about the A5/2 air-interface encryption algorithm that was used in certain GSM networks until about 2005-2007.

A5/2 was specified as a security by obscurity algorithm behind closed doors in the late 1980ies. It was intentionally made weaker than it's (already weak) brother A5/1. The idea was to sell only equipment with A5/2 to the countries of the eastern block, while the less-weak A5/1 encryption was to be used by the western European countries.

A5/2 had been reverse engineered and disclosed in the late 1990ies, and has undergone a lot of attention from cryptographers such as Ian Goldberg and David A. Wagner. In a 1999 paper, they already expect that it can be broken in real-time.

It took several more papers until in August 2003, finally, the proponents of the GSM systems (ETSI/3GPP/GSMA) have realized that there is a problem. And the problem was worse than they thought: Since they key generation for A5/1 and A5/2 is the same, a semi-active downgrade attack can be used to retroactively break previously-recorded, encrypted A5/1 calls. The only solution to this problem is to remove A5/2 from all equipment, to make sure the downgrade is not possible anymore.

Starting from 2004 the security related working groups of 3GPP and GSMA thought about removing A5/2, and in the following years they convinced their respective bodies (3GPP, GSMA), and members thereof (operators, equipment makers) to fix this problem.

Ever since that time, it is known that using the same key generation for different algorithms enables down-grade attacks. However, the key generation for the then-new A5/3 algorithm was unmodified. So now that A5/1 has been broken in recent years, even if the operators deploy A5/3, the same model of down-grading attacks to A5/1 can be done again.

I have put down a time-line at the still mostly-empty security.osmocom.org website. Some of the goodies from it:

It took from 1999-2007 until this gaping security hole was fixed. Call that incident response!
Unnamed Northern American Operators (and the PTCRB) were the biggest blockers to remove A5/2 support from their networks. This is particularly strange since US operators should always have had A5/1 access.
As a breaking of the more secure A5/1 was already anticipated even back then, in 2002 A5/3 was first specified. Five years later (2007) there was virtually no support for A5/3 among manufacturers of GSM network equipment
It took until January 2009 until the GSMA discussed A5/3 testing with mobile phone makers
It took until November 2009 until there was a plug-fest testing interoperability between A5/3 enabled GSM network equipment and A5/3 enabled phones.

And what do we learn from all this?

GSM equipment manufacturers and mobile operators have shown no interest in fixing gaping holes in their security system
Prior to that first A5/2 attack, they have never thought of procedures for upgrading the entire system with new ciphering systems (i.e. proactive plans for incident response)
Even after the A5/2 disaster, they have not learned the slightest bit. The same problem that was happening with A5/1 - A5/2 downgrade attacks can today be done with A5/3 - A5/1 downgrade attacks. And this before the majority of the operators has even started to use the already-7-year-old A5/3 in their production networks.
The security work group of 3GPP has had a lot of insight into the actual threats to GSM security even 10 years ago. You can see that e.g. in the Technical Recommendation 33.801. But nobody wanted to hear them!
Similar problems exist with the authentication algorithms. It took 12 years from first practical attacks on COMP128v1 until the GSMA is looking at withdrawing it.

All your baseband are belong to us

Harald Welte — Sat, 06 Nov 2010 19:00:00 GMT

I'd like to point out the slides of the talk: All Your Baseband Are Belong To Us by Ralf-Philipp Weinmann.

Ralf is one of those few people on this planet who have understood the security implications of now being able to send arbitrary protocol frames (particularly GSM L3 04.08 frames) to mobile phones.

GSM protocol stacks have never been written with the assumption that somebody might send intentionally malformatted messages on the air interface. But at the same time, the GSM network does not authenticate itself to the phone, i.e. everyone who can present a network-side GSM air interface to a phone will be able to exchange arbitrary messages with the phones.

This problem has been outlined in all the GSM security workshops and presentations I have been giving during recent years. Still, apart from Ralf-Philipp Weinmann's work, I have not seen a lot of public research in that area.

Exploiting and owning the baseband processor is a dangerous threat, as the microphone and entire audio path are connected to that very processor. Whoever owns the baseband can turn the mobile phone into a passive surveillance device, commonly called 'bug'. Since application processor and baseband processor are very far apart these days, with various layers of software in between, the user interface will not show any indication of what the baseband processor does.

u-boot + Linux kernel port to Mediatek MT6235 baseband processor under way

Harald Welte — Fri, 29 Oct 2010 19:00:00 GMT

I am really excited about some recent work by Marcin on starting a u-boot and Linux kernel port to the Mediatek MT6235 baseband processor.

Among GSM baseband processors, the MT6235 is a very unusual device. Unlike classic GSM baseband chips, it is not based on an MMU-less ARM7TDMI/ARM7EJS but on an ARM926EJS core. This is a full-blown ARMv5 core on which a standard Linux kernel could run.

The reason for the MT6235 to contain such an 'advanced' ARM core is simple: Mediatek is producing chipsets and reference designs for very inexpensive but feature-rich phones. Instead of going to a full-blown (and expensive) smart-phone design with separate ARM cores for the baseband and application processor, they simply make the base-band processor a bit stronger than needed for the GSM stack, and run the entire rich UI on the same cpu, including TCP/IP stack, touch-screen, web browser, e-mail client, H.264 playback / camera recording, etc.

The original firmware on the Mediatek chipsets is a Nucleus-kernel based software stack which is completely proprietary.

Now the mid-term vision for us is to have a Linux port to the MT6235, and run the OsmocomBB Layer1 (and possibly Layer2) code inside the kernel, while the Layer3 and a user interface program is running as application programs in userspace.

This would allow us to do a very rich user interface (imagine network monitoring modes, protocol tracing, manual cell selection, etc.) while still having to care only about one processor in the system. Furthermore, there are millions of MT6235 based devices, so there will be no shortage of inexpensive hardware to run this code on.

The MT6235 also has a built-in SD/MMC controller (for storing e.g. protocol traces that you take from the GSM network) and it has a fast, dual-mode USB2 high-speed USB controller for connecting it with a PC

Sure, porting our Layer1 to a completely different baseband chipset will be a lot of work, and I don't really have any idea how long it will take us. But I think the vision of such a powerful device (and finally bringing OsmocomBB and the Linux kernel together) should prove a very attractive motivational factor.

This also means: Even if you have no clue about the GSM protocols, you can now start to contribute to OsmocomBB: A lot of Linux kernel drivers for e.g. SD/MMC, USB, frame-buffer, SPI, I2C, PWM and other integrated controllers of the MT6235 need to be written.

Like all Mediatek data sheets, the MT6235 data sheet describing all those peripherals can be found on various places on the Web, including (but not limited to) Chinese developer forums.

It also seems there is at least one MT6235 based phone where JTAG and serial console have been identified (Sciphone Dream G2), which should make debugging and bootstrapping convenient.

Job Offer: GSM baseband security research in Berlin/Germany

Harald Welte — Wed, 22 Sep 2010 19:00:00 GMT

If you're following my blog because you have an interest in GSM security related topics, there is currently a very interesting job offer by Frank from GSMK. It is about a job doing research into over-the-air attacks against baseband firmware in real-world GSM/UMTS telephones. Whoever gets the job will likely use/extend OpenBTS, OpenBSC or other GSM foss projects.

So if you're familiar with the GSM/3G protocol specs, have an interest in software security / exploiting and may even have existing exposure to OpenBSC, OpenBTS or OsmocomBB: Please send Frank a message and apply for what I personally consider one of the most exciting opportunities in the IT security industry today.

Worlds first 20 minute voice call from a Free Software GSM stack on a phone

Harald Welte — Fri, 13 Aug 2010 19:00:00 GMT

As Dieter Spaar has pointed out in a mailing list post on the OsmocomBB developer list, he has managed to get a first alpha version of TCH (Traffic Channel) code released, supporting the FR and EFR GSM codecs.

What this means in human readable language: He can actually make voice calls from a mobile phone that runs the Free Software OsmocomBB GSM stack on its baseband processor. This is a major milestone in the history of our project.

While Dieter has been working on the Layer1 TCH support and the setup of the voiceband path in the analog baseband chip (audio ADC/DAC), Andreas Eversberg has been quietly working on getting call control of Layer3 into a state where it can do all the signalling required for mobile-originated and mobile-terminated call.

Combining both of their work together, they have been able to make a 20 minute long voice call from a baseband processor running a Free Software GSM stack. For all we know, it is the first time anything remotely like this has been done using community-developed Free Software. Five years ago I would have thought it's impossible to pull this off with a small team of volunteers. I'm very happy to see that I was wrong, and we actually could do it. With less than half a dozen of developers, in less than nine months of unpaid, spare-time work.

Sure, the next weeks and months will be spent on bringing the code from alpha level to something more stable, fixing known issues and known bugs, etc. But I'm confident the biggest part of the work on the OsmocomBB stack is behind us. Big thanks to the developer team driving this project forward.

Working on a document on smartphone hardware architecture

Harald Welte — Sat, 07 Aug 2010 19:00:00 GMT

I've started to write upe some information on modern smartphone hardware architecture. It will be in a similar style to what I previously wrote on feature phones and gsm modem hardware, but with a specific focus on smpartphones, their multiple processors, memory sharing, AP/BP interface, audio architecture, etc.

I should have done this a long time ago. In fact, I think I should write more documents like that on various technical subjects. If you want to learn about low-level aspects of modern telephones, there is way too little published information out there.

Official wiki page on GSMTAP created

Harald Welte — Tue, 03 Aug 2010 19:00:00 GMT

I've come up with GSMTAP about two years ago while working on airprobe. The goal was to have something similar to what radiotap does in the wifi world: A pseudo-header that adds additional information and context that is not present in the actual message.

Initially, GSMTAP was intended to be a separate link-layer type in the pcap file format, but this would preclude its use in real-time protocol analysis. So I modified it to be encapsulated in UDP packets, which are sent and received using normal UDP/IP sockets.

Over recent years, GSMTAP has not only been integrated into multiple programs of the airprobe project, but is also understood by wireshark. OpenBTS has also decided to adopt the format and can generate GSMTAP messages for debugging purposes.

After creating OsmocomBB, it was taught how to generate GSMTAP messages very quickly, too.

So by now, at least when it comes to Free Software, it is definitely the de-facto standard for capturing/transmitting and analyzing protocol messages from the GSM air interface.

However, until now, there has never been any official "homepage" of the GSMTAP header. This has changed now, the GSMTAP homepage is now part of the OsmocomBB wiki.

Playing more with Erlang

Harald Welte — Tue, 03 Aug 2010 19:00:00 GMT

Last year I started to occasionally play with Erlang. People who know me as die-hard C coder who tries to avoid C++, Java and Python wherever possible will probably be surprised here now.

I have no intention of changing my general position on programming languages. I don't feel comfortable using something where I don't know and/or understand the immediate impact on how this code will be executed on the actual silicon.

However, if you have a need to play with anything that uses ASN.1, but particularly the aligned/unaligned PER encoding variants, then it is pretty clear that there is nothing available as Free Software that can compare to the Erlang asn1ct/asn1rt modules.

At that time last year I was doing some rapid prototyping with the RANAP protocol, and the progress was quite quick. I never had time to return to that project, so it (and my Erlang skills) were left dormant.

In recent weeks, I have picked Erlang up again - again to work on ASN.1 encoded messages: This time TCAP and MAP. While we still need the in-progress TCAP+MAP implementation in C for OsmoSGSN, there are other tasks at hand where an Erlang-based implementation might yield a much higher productivity.

So right now I'm working on a program that parses/decodes and iterates through every MAP component in a TCAP message and replaces certain fields, re-encodes the entire message and sends it off the wire. Once that is done, I think I'll actually try to do a more complete TCAP server and implement a simplistic HLR for OsmoSGSN testing.

On the recent news items about the homebrew IMSI-catcher for 1500 USD

Harald Welte — Sat, 31 Jul 2010 19:00:00 GMT

Some news sites seem to do very limited research and present it as big news that you can now build an IMSI-Catcher for a budget of USD 1500, using OpenBTS and a URSP.

Let me bring some clarity into this situation:

Fundamentally, an IMSI-Catcher is nothing special but a GSM base station (BTS) that is configured to the network country code (NCC) and mobile network code (MNC) of a commercial network operator.
In GSM, the phone has no way to authenticate and thus verify the legitimacy of the mobile network. This is like a "rogue access point" in a open (unencrypted/unauthenticated) WiFi network.
Thus, anyone who has a device that can run as a GSM base station has the ability to run an IMSI catcher.
There are two Free Software / Open Source projects for running your own GSM network, both have first been published in 2008: OpenBTS and OpenBSC.
None of those two projects are intended to be used as an IMSI-Catcher but for legitimate operation of GSM networks. However, if a user choses to configure the NCC and MNC of a commercial operator and allow "unknown/unregistered/unprovisioned IMSIs (SIMs) on his network, he will effectively have an IMSI catcher.
Such operation is in violation of spectrum usage regulations, even if you have a valid test/experimental license, since that license does not permit you to use somebody else's NCC/MNC.
Furthermore, such operation is in violation of criminal law in most jurisdictions. In Germany there is a separate offense in the criminal code, called Paragraph 317 Stoerung von Telekommunikationsanlagen, combined with Paragraph 202b Abfangen von Daten.
Furthermore, there are certainly civil claims to be made by the affected operator (and its subscriber) against anyone who unlawfully operates such a fake base station
OpenBTS and OpenBSC, as well as the problems resulting from this fake base station attack have been covered in a variety of conference presentations from 2008 through today.
Thus, there is nothing new about what has been presented at Defcon 18

Also, the theoretic basics ow how to operate an IMSI catcher are nothing new either. There are even a number of patents covering IMSI catchers, the first that I know of has been patented by Rohde & Schwarz in 2003. Also, see this blog post by OpenBTS founder David Burgess on this topic.

So all that you always needed is a bit of hardware and software to send radio waves containing messages formatted in the way how they are described in the (equally public) GSM specifications as published by ETSI and 3GPP. Commercial, proprietary systems have existed for a decade. From 2008 on, there is some Free / Open Source Software to operate GSM networks. The situation remains unchanged in 2010.

So please, remember this the next time somebody is trying to tell you that this is the latest invention since sliced bread.

Dieter Spaar has started a blog

Harald Welte — Fri, 30 Jul 2010 19:00:00 GMT

Dieter Spaar, who has been involved in various ways with both OpenBSC and OsmocomBB has just started a blog. This is good news and I hope this way he will get a bit more (much deserved) exposure on his great work.

GSM Denial of Service by flooding BTS with RACH requests

Harald Welte — Fri, 30 Jul 2010 19:00:00 GMT

At Blackhat US 2010, there was a Talk that (among other things) apparently included the subject of a RACH DoS on GSM base stations, implemented using my Layer1 of the OsmocomBB software.

As some news sites are covering this as "news": This vulnerability has been long known in the field and was - to the best of my knowledge - first demonstrated to a public audience by Dieter Spaar at the Deepsec 2009 conference in November 2009. You can get his slides.

The difficult part for many years has not been to know about the possibility of this weakness. Anyone who has read the GSM air interface specification will inevitably see that there is a limited number of RACH slots and a limited number of dedicated channels. Once you fill more RACH slots than the cell has dedicated channels, and you keep re-filling them at a higher rate than the cell can expire those dedicated channels, you have a DoS.

So rather, the difficult part was to implement it in practise, as traditionally all GSM baseband chipsets have been extremely closed, just like the very software (firmware) running on them. Today, starting from Q2/2010, it is very easy to do a proof-of-concept implementation, as we have created OsmocomBB: An Open Source baseband firmware.

Dieter Spaar's implementation predates OsmocomBB development by the better part of a year. At that time, he had to resort to binary-patching existing proprietary (binary-only) baseband firmware. So I think people should recognize his effort in doing the first practical implementation of that attack.

A real-world practical A5/1 attack using airprobe and Kraken

Harald Welte — Thu, 29 Jul 2010 19:00:00 GMT

At Blackhat USA 2010, Karsten Nohl has been presenting on a practical real-world A5/1 cracking attack. For recent years, Karsten, myself and others have been speaking at various opportunities, indicating that a practical attack using readily-available information and tools from the Internet is very possible, and that it is only a matter of time for somebody actually does it.

While Karsten has focused on the actual cryptographic attack, I've been putting in some time in projects like airprobe (a GSM receiver/decoder).

Now finally, a team of friends at the new Security Research Labs (founded by Karsten) in Berlin has put the pieces of the puzzle together.

Airprobe has been extended to fully support decoding of TCH/F (FACCH, SACCH and traffic), as well as SDCCH/SACCH control channels, and to specify the timeslot and physical channel configuration from the command line. Using this, you can

decode the AGCH, wait for an IMMEDIATE ASSIGNMENT of a SDCCH
decode that very SDCCH and wait until encryption is turned on
dump an encrypted burst where you have sufficient known plaintext
use a different program to actually recover the A5/1 ciphering key
feed that key into airprobe and decrypt+decode the ASSIGNMENT COMMAND of the TCH
use airprobe to decrypt+decode that assigned TCH/F

The external program to recover the A5/1 ciphering key is called Kraken and is also available from the SRLabs website.

So what are the limitations? Well, so far this only works on non-hopping cells with a single ARFCN. The limitations are those of the receiver hardware (and SDR software), and not really limitations of the airprobe GSM decoder or the actual software tools.

In the past I would have assumed that non-hopping and/or single-ARFCN cells are rare, but in fact we can find them even inside a big city like Berlin, from at least two of the four German GSM operators. So that's why this attack is very practical, no matter what the GSMA might say.

Implementing the TCAP protocol, heading towards OsmoSGSN SS7 support

Harald Welte — Sat, 10 Jul 2010 19:00:00 GMT

The protocol by which traditional GSM core network components interact is called MAP (Mobile Application Part). MAP itself is a user of the TCAP (Transaction Capabilities Application Part) protocol, which in turn runs on a SS7 protocol stack (i.e. SCCP over MTP or M3UA or SUA over SCTP).

For those users of OpenBSC who have a need to interoperate with other GSM networks (roaming), the circuit-switched part of OpenBSC has so far relied on the use of a proprietary MSC (by means of the A interface). This closed MSC then talks MAP/TCAP/SS7 to roaming partners.

However, on the GPRS front, we now have OsmoSGSN. However, as opposed to the BSC on the circuit switched side, the SGSN directly interacts with the core GSM network components (both of the home network and the roaming partners).

So in order to run OsmoSGSN interacting with existing HLRs, we need to add a MAP/TCAP/SS7 interface to it. Once this has been done for the SGSN, we of course can do the same for the MSC-part that is currently integrated with OpenBSC.

As there are existing implementations of SCTP (inside the Linux kernel) and SUA (sualibrary), TCAP is the next step in the protocol stack that needs to be implemented. I've been digging into TCAP for the last week(s), and believe I finally understood every part of its operation.

You can think of TCAP as something that facilitates the transport of request-response type transactions over a datagram oriented transport layer. It intends to have lower overhead than a connection-oriented service (e.g. establishing TCP sessions) and supports features such as aggregating multiple user-messages (called components) in a single actual transport-layer message. The idea is to reduce the overhead of message headers and routing.

TCAP is (unfortunately) specified in ASN.1 and thus requires significant effort to parse and construct. Right now I'm using Lev Walkin's asn1c ASN.1 C code generator to generate the parser and constructor functions. The actual TCAP protocol logic is once again implemented in plain C, using the various concepts and utility functions established in OpenBSC (and now part of libosmocore).

The implementation is making good progress and I hope I can do some early testing in about a week from now, and successively move straight to the MAP protocol, implementing at least those parts that we need for GPRS authentication and attach / routing area updates.

Major update in OpenBSC GPRS/EDGE support

Harald Welte — Thu, 01 Jul 2010 19:00:00 GMT

Through the last couple of days, I've been in extreme bug-squashing mode for the GPRS/EDGE code base in OpenBSC (mostly the OsmoSGSN program). I'm now at a point where I can reliably establish PDP contexts and access the Internet from a variety of different phones with different baseband chipsets and GPRS protocol stack implementations. All so-far-known bugs regarding fragmentation/reassembly, sequence numbering and other issues have been fixed. There definitely are plenty more, but we first need to find them.

Since it's working reliably now, it's quite fascinating what the various phones do after connecting to the GPRS network. Like Windows Mobile phones sending Netbios Name Service updates (and requests), which I think is funny considering that they are sent to a network that is typically considered to be the public Internet.

But to be fair and not anti-Windows, my Google/Android G1 also makes some https connections back to Google - and I don't know what they are for [yet].

In any case, with OpenBSC, OsmoSGSN and OpenGGSN anyone interested in doing true security (and privacy) research with mobile phones is now able to do so. Using those programs, you can run your own GPRS+EDGE network and can see first hand what your phones are doing on a cellular network, what kind of data they are sending back home. In this setup, there is no packet filtering, NAT, deep packet inspection and no intrusion detection systems between your PC and the IP stack on your phone.

The reason why you see paging by IMSI in real-world GSM networks

Harald Welte — Sun, 27 Jun 2010 19:00:00 GMT

During my work on airprobe and OsmocomBB I've been wondering why you see paging by IMSI in real-world GSM networks.

A quick recap: The IMSI is the world-wide unique serial number of your SIM. Since it is easy to identify and track people, the TMSI was introduced as a temporary identifier that is frequently re-allocated over encrypted channels. The only reason for the TMSI to exist is to prevent tracking of a subscriber by watching where his IMSI appears on the paging channel.

According to the theory, the IMSI is only used when first registering to any GSM network. At that time, a TMSI is allocated to the SIM card in the phone, and this TMSI is used for the next transaction(s). Later, this TMSI is re-allocated and re-allocated, but the IMSI shouldn't show up again in any paging requests.

Even if you switch mobile networks (i.e. in the roaming case), you would once send the IMSI as part of a LOCATION UPDATE REQUEST or IDENTITY RESPONSE, but the network has no need to page the SIM by IMSI.

So far the theory. If you look at the Paging Channel (PCH) of cells in real-world networks, you see a significant (10-20%) amount of paging requests that contain paging by IMSI. This seems strange on first sight, given the theory described above.

I have the following plausible explanation for this:

The VLR keeping the IMSI-TMSI mappings doesn't have non-volatile storage. This means at a VLR restart, all the TMSI allocations will be lost, and the network has to resort to paging by IMSI.
The VLR has a limited amount of RAM, which can store a limited number of IMSI-TMSI mappings. Especially if the operator is interested in saving money, the amount of memory is insufficient for all subscribers in the network. This means, the VLR will expire some old entries in the mapping table to store new entries. Thus, mobile phones whose last transaction with the GSM network was relatively long ago are likely candidates for such VLR expiration. Once a phone for an expired entry needs to be paged again, paging will happen by IMSI.
Last, but not least: GSM networks do not page a phone by the last known cell, but by the last known location area of the phone. A location area might be relatively big. This means that at any cell you will see a lot of paging messages, even for phones that are not even anywhere near this cell. If there is no response within the location area, the MSC might decide to do paging on a larger radius, possibly the entire MSC area. Since such MSC-wide paging is likely to occur for phones that haven't shown activity for a long time (and thus might have moved or disappeared without properly unregistering from the network), those are the exact same phones for which the IMSI-TMSI mappings have expired from the VLR. Thus, the rate of paging-by-IMSI looks disproportionately high.

So the relatively high percentage of paging by IMSI vs. TMSI should not be taken as a measurement with regard to the total number of transactions or even the total number of subscribers. It is simply the mechanics of the network resulting in a distortion of those figures caused by phones that have never properly unregistered from the network.

Back from OpenBTS workshop

Harald Welte — Sat, 26 Jun 2010 19:00:00 GMT

I've just returned back from the First OpenBTS workshop held by David Burgess and hosted by Dieter Spaar in south-east Bavaria (Germany). While I'm not involved with OpenBTS so far (except from using it occasionally), I still thought the community surrounding Free Software / Open Source in the GSM field is small enough to make me participate.

On the request of the participants, I also did a short demonstration of both OpenBSC and OsmocomBB. And just like I managed to crash OpenBTS by accidentally sending invalid messages, my OpenBSC demo crashed at some point [due to a not-yet-known bug regarding SMS delivery. I suppose the intrusive changes of the BSC/MSC split are to be blamed for that. But I don't mind, we need that split...

I definitely had a great time meeting the participants of the workshop. There definitely is a very diverse crowd with equally diverse reasons for their interest in using and/or deploying OpenBTS.

Finally, there was a chance to discuss the need for a common 'application interface' in both OpenBSC and OpenBTS. Using that interface, external applications (e.g. implementing USSD or RRLP) could be written in a way to work with both OpenBTS and OpenBSC. I hope we can get started on this soon and remove another bit of fragmentation in what is already a fairly small special interest community...

Given the excellent weather conditions, the motorbike ride to and from the venue went fine - despite being at 650 km distance from my home.

Adding frequency hopping support to OpenBSC

Harald Welte — Sat, 19 Jun 2010 19:00:00 GMT

During the last couple of days, I've been adding the bits required to support frequency-hopping BTSs in OpenBSC. Now everything looks great in the protocol traces - but unfortunately it still doesn't work, at least not with the Siemens BS-11 that I have access to.

Will continue to try to make it work. The big advantage of having a hopping BTS under our control is that we can define the hopping sequence - something quite useful once we get to the point where we'd like to add frequency hopping to the telephone-side stack (OsmocomBB).

The good news is that I had to fix lots of bugs in the A-bis OML dissector for wireshark that I wrote some time ago. It's now much more complete and definitely a big step further towards eventually getting it included in wireshark mainline.

A fairy tale about ICCIDs, IMSIs and iPads

Harald Welte — Mon, 14 Jun 2010 19:00:00 GMT

One of the big news of the last week is AT&T's leak of 114,000 iPad customer records including the e-mail address and ICCID

While that leak is certainly a big issue in itself, there are some people, most notably Chris Paget, who claim that this is much more serious than generally assumed. The main claim here seems to be that ...in order to translate an ICCID into an IMSI, you need to query the HLR.

I have been reading GSM protocol specifications on every level for the past years, and never have I seen the ICCID being mentioned anywhere. The GSM specifications do not require this information to be stored in the HLR, and the MAP protocol (used on the C interface between MSC and HLR, see 3GPP TS 29.002) does not even know how to encode/specify it.

Also, there is no technical need for it. The ICCID is never used nor needed in any part of the GSM protocol. Also, the GSM network typically doesn't store any information that is not absolutely necessary for its operation. The only identifier of a SIM card that the network protocols care about is the IMSI.

So unless the US operators in question have either some kind of proprietary extensions to both their HLR and the MAP protocol, there is to the best of my knowledge no way how you can relate the ICCID to the IMSI.

And thus, as a result, the IMSI-catcher attack described will not work since you don't know the IMSI of the SIM card (associated with the customer record) that you want to catch.

If anyone can show me hard technical facts about ICCIDs being used in the HLRs of the operators in question, I am happy to post here I was wrong. Otherwise, I would hope everyone else could also come down to the hard technical facts, i.e. which particular MAP message is used for this alleged ICCID-to-IMSI query.

UPDATE: As some people have discovered, the three US operators themselves have decided that they use the same number to generate both the ICCID and the IMSI. So if you have one, you can compute the other. No need for HLR access, no need for the MAP protocol. So the information leak is in fact unrelated to the GSM protocol but simply a matter of how unfortunate those particular three operators assign their unique identifiers.

Wanted: Packet traces of the MAP+TCAP based C/Gc interface

Harald Welte — Sat, 05 Jun 2010 19:00:00 GMT

I'm looking for any example pcap files (packet traces) of the so-called "C" and "Gc" Interfaces, i.e. the interfaces of the HLR (Home Location Register). If anyone has such pcap files or can generate some, I would very much appreciate it.

The protocol levels I'm interested in is SCCP, TCAP and MAP. The lower layers (MTP) are not important now.

Specifically, I'm looking for traces of any of the following MAP operations:

updateLocation
cancelLocation
restoreData
sendParameters
updateGprsLocation
sendAuthenticationInfo
purgeMS
sendRoutingInfo
sendRoutingInfoForSM
reportSM-DeliveryStatus
readyForSM
noteSubscriberPresent
sendRoutingInfo
anytimeInterrogation
statusReport
{register,erase,activate,deactivate,interrogate}SS
sendIMSI
sendRoutingInfoForGprs
insertSubscriberData
deleteSubscriberData
checkIMEI

If anyone is able to produce the respective traces, I would appreciate it a lot. I'd need them as examples to make sure I fully understand the TS 09.02 in combination with Q.77x before actually starting to implement it...

First functional HTTP transfer in my own GPRS/EDGE network

Harald Welte — Wed, 02 Jun 2010 19:00:00 GMT

Today marks the day where finally, after months of (non-full-time) work, I have made the first successful HTTP connection through my own GPRS/EDGE network.

Ever since we started to seriously get into OpenBSC to run GSM networks, I've been looking forward to running GPRS networks, too. What most people don't know: GPRS is radically different from GSM. It basically only shares the frequencies and timeslot architecture of the physical layer, while having it's own layer1, layer2 and various other protocol layers. Also, its signalling and data completely bypass the usual BSC and MSC components of a GSM core network.

So what I've been working on is now called OsmoSGSN. Using OpenBSC, you can provision an ip.access nanoBTS (or any other BTS with a Gb Interface) to broadcast the GPRS/EDGE capabilities to the handsets. The BTS then establishes the Gb interface (consisting of NS and BSSGP) to the SGSN.

The SGSN takes care of GPRS Mobility Management (GMM) and Session Management(SM) in the signalling plane, as well as the LLC + SNDCP protocol layers. On the other end, it uses the GTP protocol to connect to a GGSN. In our case, this is the OpenGGSN project which I recently adopted.

OpenGGSN then creates a virtual network device (tun0), through which the actual IP packets are entering/leaving the GPRS network. From there you can route and/or NAT them just like any other IP packets.

The current code is still incomplete in many places and known to be unstable. But it's really rewarding that after a long time of development, layer after layer of the stack, finally actual TCP/IP can be provisioned to phones.

The code is in the current master of the openbsc git repository, but I don't think there's much point in trying it just yet. I suppose in a week from now things should be much more stable.

OpenGGSN Version 0.90 released

Harald Welte — Mon, 24 May 2010 19:00:00 GMT

Only three weeks ago I blogged about OpenGGSN, a seemingly-abandoned Free Software implementation of the GGSN node of the GPRS core network.

Things have developed quite a bit ever since. As the original author didn't respond to any of my mails and sourceforge.net was not able to reach him either, they have approved my the 'abandoned project takeover' (APT) request.

I've now switched the project from CVS to git, removed links to the non-existing openggsn.org homepage and released version 0.90, containing nothing less than a fix for remote DoS vulnerability that was pending for more than 5 years.

So far I'm only exercising the PDP context activation/deactivation parts of OpenGGSN from OsmoSGSN (the SGSN I write as sister-project to OpenBSC), but I hope we can use OpenGGSN in a production GPRS network soon...

Doing even more encapsulations than the GPRS Gb interface already has

Harald Welte — Tue, 18 May 2010 19:00:00 GMT

Back in October 2009, I blogged about the incredibly deep protocol stack on the GPRS Gb interface between a BSS and the SGSN.

Today I had the pleasure of implementing an even more odd variant of the Gb interface, where NS does not get encapsulated in UDP/IP/Ethernet, but in FrameRelay/GRE/IP/Ethernet. The total protocol stack thus then looks like: HTTP/TCP/IP/SNDCP/LLC/BSSGP/NS/FR/GRE/IP/Ethernet, with an optional PPP between IP and SNDCP. If anyone does the math to calculate the total protocol overhead, please let me know[...

The reason for that oddity is apparently that there are Cisco and other routers that can encapsulate Frame Relay in GRE. So using a old circuit-switched SGSN with E1 interfaces and such a router, you can convert from Frame Relay on E1 to Frame Relay on GRE/IP/Ethernet.

Both the Gb Proxy as well as the upcoming OsmoSGSN use the same NS implementation, i.e. they can now both talk NS/FR/GRE and the NS/UDP variants - even at the same time, as the encapsulation is a property of each individual NS-VC.

Back from a week of GSM/GPRS protocol coding/testing in Iceland

Harald Welte — Sat, 15 May 2010 19:00:00 GMT

With only 16 hours delay (which isn't all that much considering the volcanic ash situation) I arrived back in Berlin from one week of OpenBSC software hacking, particularly on the GPRS side of things.

It was really nice to see to what extent OpenBSC software is already used at On-Waves, providing GSM and now also GPRS services to thousands of users.

My work was mostly focused on the Gb-Proxy, a multiplexer/proxy for GPRS Gb links running the NSIP (NS-over-IP) protocol. It combines elements of the idea of a network address translator with that of a proxy, combined with a little bit of packet-based routing. This really makes me feel like I'm back to packet-switched networking, which is great. Especially the fact that we use the VTY code from quagga and its interactive command line sometimes lets you forget that you're not working with classic TCP/IP routing daemons or the like ;)

Aside from that, I continued my work on the upcoming OsmoSGSN, using which we will be able to run an autonomous GPRS network with no dependency on external proprietary components. In this setup, the PCU in the BTS connects over Gb/IP to OsmoSGSN, which then talks over GTPv1 to the OpenGGSN.

Also, work was spent on an abstract rate_counter implementation (now part of libosmocore). The idea is to have a counter that will count certain events (like number of packets/bytes, number of link failures, etc), but also keep a small history about how many of those events happened in the last second, last minute, last hour and last day. There is also common code to store those counters in the database, as well as to print them on the VTY. The new counters are so far only used in the GB-Proxy, but they will soon likely be added to OpenBSC (bsc_hack) and other programs of our Free Software GSM network portfolio.

Heading for 4 days of Iceland to work on OpenBSC GPRS

Harald Welte — Sun, 09 May 2010 19:00:00 GMT

Having just returned from Croatia the day before yesterday, I'm about to head on a four-day trip to Iceland, where I'll be doing some testing and bug fixing of the current OpenBSC GPRS support at On-Waves.

Zecke is also going to be there, working on other aspects of OpenBSC. This will make the trip even more fun!

CECT C3100: Not a phone, but a flashlight with integrated phone

Harald Welte — Sat, 08 May 2010 19:00:00 GMT

I've seen many [mobile] phones in my life, but nothing like the CECT C3100 so far. It's made of the cheapest hard plastic, like cheap kids toys. In addition to the phone keyboard, it has a mechanical switch on its side. If you slide that switch, five powerful bright white LEDs at the top of the phone will turn the entire device into a flashlight.

However, it is one of the most basic phones with one of the older/simpler MTK baseband chips inside (MT6223). Also, as we have determined by a PCB delamination analysis, the test pads next to the MT6223 really are its ARM JTAG pins.

JTAG is something not commonly found in MTK phone designs, but it is definitely a big win for bootstrapping any system-level software such as drivers on the unit.

Right now I don't have the time to work on MT6223, we still have many issues to fix in the current Ti Calypso code. But I can't wait to find time to see if we can extend our hardware support to MTK GSM chipsets...

GPRS progress in OpenBSC

Harald Welte — Mon, 03 May 2010 19:00:00 GMT

In recent weeks, I have been able to pick up my work at GPRS support for OpenBSC again. What has been done is:

Add OML support to configure nanoBTS for EDGE
Add RR (System Information) support to indicate EDGE support
Make a OpenBSC + nanoBTS setup inter-operate with an existing SGSN
Develop a proxy that can aggregate the Gb-interfaces of multiple BTS into one Gb link to a real SGSN. This way the SGSN has only one Gb link for all the cells under the control of a BSC, as opposed to one Gb link for each and every BTS

What I'm working on now is the actual SGSN implementation. The SGSN is mainly responsible for GPRS mobility management (GMM) and for terminating the Layer2 (LLC) protocol from the MS. This is very different from circuit-switched GSM, where Layer2 (LAPDm) already terminates at the BTS.

The layering stack of GPRS is a real nightmare, I am sure I have indicated this in this blog before. The Current OsmoSGSN code (available from the regular openbsc.git repository) implements the NS, BSSGP and LLC layers, as well as the basic GSM 04.08 GPRS Mobility Management messages like GPRS ATTACH/DETACH and ROUTEING AREA UPDATE. The LLC code is still somewhat limited, but for the time being it is sufficient.

What drove me crazy for a couple of days is the number of parameters that are exchanged at PDP CONTEXT ATTACH time. There are no less than 26 different quality of service (QoS) parameters negotiated (see struct gsm48_qos at the bottom of this link), each of them from a wide range of values. It's almost impossible to imagine more than 1% of all the possible combinations have ever been used in production networks. The QoS parameter negotiation works by the phone sending a list of requested parameters, to which the SGSN responds with its selected parameters. My first thought was: Lets be smart and simply echo back the QoS parameters - the phone must accept what it has requested. That didn't work either: While the QoS structure is the same in both ways, the actual values in uplink and downlink directions are encoded differently. Who on earth defines such an encoding?

Next item was the XID exchange which is at the boundary between LLC and the SNDCP (Sub-Network Dependent Convergence Protocol). It works like this: The phone proposes an endless list of parameters, which the SGSN can evaluate, and then depending on the parameter type either negotiate up or down. According to the spec, sending an empty XID response should mean "I agree with all your parameters". However, at least those phones that I tested were not happy with that. So I decided to simply send back the entire XID block to the phone. And believe it or not, as opposed to the QoS parameters, this time it even worked

So now I'm facing the implementation of the actual SNDCP-to-GTP interworking, which is nothing less but the guts of the SGSN. GTP is the protocol used on the GGSN side. At least this time, GTP is sent directly over TCP or UDP, i.e. the stacking inside the SGSN is only one layer deep, while on the Gb-interface it is four (NS,BSSGP,LLC,SNDCP).

SNDCP interacts with the GPRS Mobility Management, GPRS Session Management (both GSM 04.08 over LLC), the GTP interface to the GGSN, as well as other parts. I expect many pitfalls on the way to getting it working, and given the complexity involved I have already decided to stick much closer to the specification than I usually did with the OpenBSC work. This means properly implementing all the state machines with all their transitions, exceptions and timers. I'm sure it's going to be "fun". The good part of it is: Most of the SGSN will be re-used once we finally get around adding support for 3G/UMTS/WCDMA cells.

OpenGGSN - An open source GGSN implementation

Harald Welte — Mon, 03 May 2010 19:00:00 GMT

As a friend pointed out to me at exactly the right point in time: there already is an Free implementation of a GGSN. In case you don't know what a GGSN is: It is one of the two core components of a GPRS network. So, in order to extend a OpenBSC GSM network with GPRS support, there are two components required: The SGSN (on which I'm working currently, project name OsmoSGSN), and the GGSN. Due to the good news about OpenGGSN, I'm quite confident that I will not need to implement the latter part.

OpenGGSN is not only a Free Software implementation of the GGSN, but it is also licensed under GPLv2, making it compatible with the OpenBSC codebase (which is GPLv2 or later). This means I will be able to link the OpenGGSN-provided libgtp library (implementing both sides of the GTP protocol between SGSN and GGSN) from OsmoSGSN, further reducing the amount of work required to get this working.

However, despite seeming like a fairly advanced/complete implementation of the GGSN specification: OpenGGSN seems like a project that was abandoned many years ago. The latest CVS commit is from 2005, and all of the bug fixes that people have submitted to the bug tracker have not been merged. The homepage is defunct, and the openggsn.org domain name seems to have been expired (and grabbed).

I've tried to contact the author by e-mail about his intentions for the project, let's see if there is any response. Meanwhile, I have generated a git repository from the OpenGGSN CVS repository at sourceforge and applied all the pending fixes to a local branch. See my related mailing list post for more information.

Working on GPRS support for OpenBSC again

Harald Welte — Mon, 26 Apr 2010 19:00:00 GMT

This has been on my TODO list for at least the last six months or so: Growing the experimental GPRS branch of OpenBSC into something more useful.

Right now, you can use OpenBSC with a GPRS-capable BTS - but only if you have an existing SGSN to serve the Gb interface of the BTS. This somehow defeats the point. We want to offer a 'GSM network in a box' solution, where no other non-free Software is required to run a fully functional small network.

So right now I'm cleaning up the 08.16 (Network Services) Implementation, and will move my way up through the existing 08.18 (BSSGP) and LLC code that I wrote some time ago.

With some luck, in a couple of weeks we should be able to run a self-sufficient combined GSM + GPRS (+ EDGE) network out of OpenBSC.

Paper: Anatomy of contemporary GSM cellphones

Harald Welte — Tue, 13 Apr 2010 19:00:00 GMT

During the last days, I was working on an introductory paper on how a GSM cellphone actually works. It is titled Anatomy of contemporary GSM cellphone hardware and should provide a good technical text for anyone who generally is into technology and understands a bit about both software, computer architecture as well as radio, but who still feels like he has no clue what is actually happening inside the phone, particularly the hardware side.

The text does not cover the GSM protocols itself, as there is much more information available on this already.

Feel free to let me know what you think, I'm always happy to extend or clarify it based on your feedback. I hope some people find the text useful.

OpenBTS workshop in June (Pfarrkirchen/Germany)

Harald Welte — Sun, 28 Mar 2010 19:00:00 GMT

If you've always been interested in Open Source / Free Software GSM using OpenBTS, you might be interested in the upcoming three-day OpenBTS workshop in Pfarrkirchen/Germany.

The workshop is held by OpenBTS project leader David Burgess, and will cover subjects ranging from OpenBTS installation, configuration and deployment down to actually extending OpenBTS with some custom code.

p.s.: Please don't misunderstand, the workshop really is about OpenBTS not OpenBSC - though there might be OpenBSC folks hanging around ;)

OsmocomBB now performing location updating procedure against GSM cell

Harald Welte — Thu, 04 Mar 2010 19:00:00 GMT

I haven't had much time for blogging recently, too much exciting work going on at OsmocomBB:

we now have simplistic support for Uplink (transmit) on SDCCH/4
we have a minimal Layer2 (LAPDm) implementation
we can send LOCATION UPDATING REQUEST to the network, and receive the respective response
there's wireshark integration, i.e. all packets on the L1-L2 interface can be sent into wireshark for protocol analysis

There are still many limitations, but this is a major milestone in the project: We have working bi-directional communication from the phone to the network!

The limitations include:

The cell has to use a combined CCCH (SDCCH/4 on timeslot 0)
The cell has to use no encryption/authentication
The layer2 is not finished, especially re-transmissions will not work yet
There's no power control loop yet
There's no timing advance correction

However, most of those are more or less simple we know what needs to be done, its just a matter of getting it done kind of tasks. There are no big unknowns involved, and particularly no further reverse-engineering of the hardware is required.

Also, the existence of a stable bi-directional communications channel between the network and the phone means that anyone interested in working on the higher layers can now actually do so. Completing and testing layer2 as well as RR/MM/CC on layer3 is a major task in itself, and it definitely requires the lower layers to be there.

The other good part is that development of layer2 and layer3 can happen entirely on the host PC, where debugging is much easier and there's no need for cross-compilation and we can use all the usual debugging options (gdb, valgrind, ...)

I'm now almost heading off for holidays (starting March 10), so don't expect any major progress from me anytime soon. I hope other interested developers will be able to take it from here and fill in some missing gaps until I'll get back.

Looking for documentation on sunplus SPMA100B

Harald Welte — Sun, 28 Feb 2010 19:00:00 GMT

In the Motorola/Compal C155 phone supported by OsmocomBB, we have found a ringtone melody chip called SPMA100B from sunplus.

As strange as it might seem, this is the only part used in the phone for which we have not been able to find any kind of programming information. So if you know anything about how to program this part from software (register map, programming manual, ...) please let me know!

And no, we don't need electrical/mechanical data sheets, thanks :)

Restructuring OpenBSC and OsmocomBB code

Harald Welte — Fri, 19 Feb 2010 19:00:00 GMT

I've spent the better part of the day with , renaming files/functions/include paths, Makefiles, autotools and the like.

The result of this is a new sub-project called libosmocore that gathers all the shared code between the network-side GSM implementation OpenBSC and the phone-side implementation OsmocomBB. The library is portable enough that it can run on a proper OS (like GNU/Linux) but also be cross-compiled to work on the actual phone without any OS.

On the other hand we now have a master Makefile in OsmocomBB to build libosmocore for host PC and target (phone), as well as the osmocon and layer2 host programs and the phone firmware itself.

Let's hope I can now return to writing actual code...

Announcing OsmocomBB: Free Software / Open Source GSM Baseband firmware

Harald Welte — Thu, 18 Feb 2010 19:00:00 GMT

Last, but not least, I am proud to announce the OsmocomBB project publicly. During the last 7 weeks, a small group of skilled developers has been working on this

It has now reached a point where we can

scan the spectrum for the strongest signal GSM channels
lock onto them and performing AFC (automatic frequency control)
decode the SCH burst to obtain BSIC and GSM frame time
decode the BCCH of the cell, pass it over to the host PC and feed it into wireshark

Since this in itself is a valuable and useful milestone of the project, it was the ideal opportunity to take this project public.

There's still a lot of work to be done in many areas. Most of them are not even related to the GSM air interface. So if you're familiar with C development on an ARM7TDMI based microcontroller, know your way around I2C and SPI, are familiar with the GNU toolchain for ARM and want to help us out: Please join the baseband-devel mailing list right away!

In six weeks from bare hardware to receiving BCCHs

Harald Welte — Fri, 12 Feb 2010 19:00:00 GMT

After six weeks of full-time hacking, with the help of a few friends, we have made it to receiving actual BCCH data from a GSM cell.

So what does this mean? As I have indicated publicly at the 26C3 conference: Now, that we have managed to create a working GSM network-side implementation (OpenBSC) during the last year, we will proceed to do the same with the phone side.

Initially we spent quite a bit of thinking on building our own custom hardware. But while planning for the first prototype, we realized that it would simply distract us too much from what we actually wanted to do. We don't want to take care of component sourcing, prototype generations, quality assurance in production, production testing, etc. -- All we want is to write a Free Software GSM protocol implementation for a phone.

Unfortunately (as usually in the industry), the silicon and device makers do not publish sufficient documentation about their devices to enable third-party developers to go ahead and write their own software: The never ending problem of Free Software in many areas beyond more-or-less standardized hardware like in the PC industry.

So, if you want to write Free Software for such a device, you have two options:

Reverse engineering the existing hardware and writing your code based on that information
Building your own hardware and then writing the software you wanted to write.

I've been involved in both approaches multiple times while looking only at the application processor (the PDA side) of mobile phones: OpenEZX and gnufiish are two more or less abandoned projects aimed at reverse engineering. Openmoko was the project that had to build its own hardware as a dependency to be fulfilled before writing software.

If you're not a company and don't want to sell anything, the reverse engineering approach looks more promising. You can piggy-back on existing hardware, don't need to take care of sourcing/production/certification/shipping and other tedious bits.

If you are a company and want to generate revenue, then of course you want to build the hardware and ship it, as it is what you derive your profits from.

So, just to be clear on this: Neither OpenEZX, nor gnufiish nor Openmoko were ever about writing Free Software for the GSM baseband processor, i.e. the beast that exchanges messages with the actual GSM operator network. But this is what we're working on right now.

It's about time, don't you agree? after 19 years of only proprietary software on the baseband chips in billions of phones, it is more than time for bringing the shining light of Freedom into this area of computing.

To me personally, it is the holy grail of Free Software: Driving it beyond the PC, beyond operating systems and application programs. Driving it into the billions of embedded devices where everyone is stuck with proprietary software without an alternative. Everybody takes it for granted to run megabytes of proprietary object code, without any memory protection, attached to an insecure public network (GSM). Who would do that with his PC on the Internet, without a packet filter, application level gateways and a constant flow of security updates of the software? Yet billions of people do that with their phones all the time.

I hope with our work there will be a time where the people who paid for their phones will be able to actually own and control what it does. If I have paid for it, I determine what software it runs and when it send which message or doesn't.

Oh, getting back to what our work: It will be published as soon as it is sufficiently stable and fit for public consumption. You won't be able to make phone calls yet, but we'll get there at some later point this year.

Symbian is Open Soruce - Really?

Harald Welte — Thu, 04 Feb 2010 19:00:00 GMT

In recent news, the Symbian Foundation announced that "All 108 packages containing the source code of the Symbian platform can now be downloaded from Symbian's developer web site". This is great news!

This morning I tried to look at the parts most interesting to me: phonesrv (implementing call engine, cell broadcast and SIM toolkit APIs) and poc (implementing push-to-talk). Their pages don't have the usual "source code" tab at the bottom right which links to mercurial and tarball download pages!

Either I'm too stupid, or I am unable to find any source code for those two components. I'm quite sure something essential like the API's for making phone calls are considered part of the Symbian platform. So how does that match with the statement that all packages containing the Symbian platform can now be downloaded?

Sorry for new blog updates

Harald Welte — Tue, 26 Jan 2010 19:00:00 GMT

I've been busy day and night, hacking away on my latest pet project in the GSM field. In fact, it's been a long time since I've been able to dedicate so much time and energy into one particular project, without many distractions at all. The project is now finally looking quite promising and making nice progress throughout the last three weeks.

If progress continues, I hope in another week I'll be able to reveal what this is all about. I haven't felt this level of excitement since the early days of Openmoko :)

Illusions about MagicJack at CES

Harald Welte — Thu, 07 Jan 2010 19:00:00 GMT

Many people have pointed out the MagicJack Femtocell product that has been announced at CES. I cannot really understand the big hype and news about it. Why? read further...

On the technical side, there is hardly anything new. Using projects like OpenBTS or OpenBSC, you can run your own GSM network and connect it to VoIP. Sure, the retail price of the MagicJack is much lower, but that's the economics of scale. As soon as OpenBSC support for one of the recent femtocells is done, we also have a much lower cost solution to the same problem.

On the legal and business side, I can see many problems for MagicJack:

To operate equipment in the GSM or 3G spectrum, you need a license. Since a nationwide GSM operator license is very expensive in about any country of the world, it is economically not an option. Selling the MagicJack devices without a license and leaving the spectrum license to the user will not work, or at least not long, since regulatory authorities and commercial operators are not going to let anyone deploy systems that interfere with their networks.
If you keep the Operator's SIM in the phone, and use that SIM on your own network you might at least violate the terms of services of the operator. The SIM card normally belongs to the operator, and it is part of the users business relationship with that operator. As such, you can not really use it with other networks. Sure, if you do this at home with your OpenBTS/OpenBSC installation, nobody will care. But if somebody is doing this commercially, and in a way that affects the sale of talk time of the regular operators, again it will not take long until the commercial operator will sue you.
Security. If you run your phone with a "foreign" SIM, you do not know the Ki on the SIM and thus cannot do cryptographic authentication and/or encryption. This is a big security issue. It is once again not an issue in your personal testbed setup, but it is going to be one if you do this at large scale as a mass market product.

So, as you can see: It's neither technically something exceptionally new, nor is it something that has a very promising business or legal outlook. The only way how a product like this would work is if it is authorized by the respective operator. But why would the operator authorize something that will take talk time away from his network and thus his revenue stream?

Planning for a GSM development board

Harald Welte — Wed, 06 Jan 2010 19:00:00 GMT

I've finally found enough spare time to work on detailed plans for a GSM development board. The idea here is to have a 100% open hardware design with 100% free software that provides an inexpensive platform for GSM related R&D.

Initially the focus is on having a board that can behave like a GSM cellphone, next steps would be to have a multi-channel frequency-hopping receiver, and finally the option of using it as a BTS.

The idea is fairly simple: Take a commercial off-the-shelf analog baseband and RF circuit for GSM and attach it to a general purpose DSP, add some glue logic and go ahead. But the devil is in the details:

You want something where you can still find the parts on the market, but which still has sufficient leaked documentation that you can write an open source driver for it.
The requirements for a MS and a BTS are quite different. A phone never needs to continuously receive and transmit in all timeslots, e.g.
The requirement for a multi-channel frequency-hopping receiver is mainly a high number of receiver circuits, so the solution needs to scale to a larger number of receivers.
The analog baseband circuits often have quite obscure control interfaces which need to be driven. Custom peripherals in programmable logic (CPLD/FPGA) are required.
The TDMA nature has strict timing requirements. Normal processors and software-based mechanisms are not sufficient to trigger the required events and their sequencing at a high enough precision.

Anyway, there is sort of a first plan now, and the next weeks and months will be spent with refining the plans and building a first proof-of-concept prototype. Once that has proven to work, we want to go ahead with finishing the design for a real board, to be manufactured in sufficient quantities for interested parties.

Unless you have worked in GSM phone or base station hardware design or have a similar level of EE and DSP skills, please refrain from contacting me right now about how to join the project. We want to focus on getting things going first, then make a public release at a point where there is something that works sufficiently well that we can support a larger community of developers.

2010-01-02 / 2pm CET: Radio Interview at Deutschlandradio

Harald Welte — Fri, 01 Jan 2010 19:00:00 GMT

The German radio station Deutschlandradio Kultur has invited Constanze Kurz (46halbe) and myself for interviews during today's Breitband radio show. We'll be talking about the 26C3, the Chaos Computer Club and - of course - GSM [in]security.

OpenBSC now has handover support

Harald Welte — Sun, 20 Dec 2009 19:00:00 GMT

So far, OpenBSC already implemented mobility management, i.e. keeping track of which location area a mobile phone is locate in. However, this only works during idle mode, i.e. while there is no actual phone call in progress.

Throughout the last week, I've been working on getting real handover support into OpenBSC. This is now actually working very well! You can move from one cell into another cell while your phone call continues just like it is supposed to do.

The signalling part is actually not all that hard to implement. However, it has some dependencies on things like measurement reports, which in turn require us to send proper neighbor lists, which in turn requires proper generation of system information messages, etc.

The actual order of events in a successful handover case is as follows:

OpenBSC send the neighbor cell information in the BCCH and SACCH
OpenBSC regularly receives measurement reports from the phone, where it tells us how well neighbor cells are being received. OpenBSC then averages those measurements and decides
allocates a channel on the new BTS
waits until the BTS acknowledges this
sends a HANDOVER COMMAND to the phone through the old BTS
waits for HANDOVER ACCESS bursts and a HANDOVER COMPLETE on the new BTS
closes the old channel on the old BTS
takes care of re-routing the voice channels

As indicated, the signalling part was relatively easy, and once the measurement processing and neighbor lists were in place, this worked really quick. What turned out to be a much bigger PITA was the handling of the actual voice streams.

Let's assume you have a call from A to B, where B is changing cells and now becomes B*. In this case, after switching cells, the speech frames from A need to be re-routed to B* instead of B. That's simple and works very easy. In the other direction, switching off B is easy. However, a completely new channel B* suddenly sends speech frames to A. In case of classic TRAU frames on E1 that is simple as they don't have any context.

In the case of ip.access nanoBTS, the speech frames are transported using RTP. Changing the source of your stream will change its CSRC (synchronization source identifier), timestamps and sequence numbers. The receiver in BTS A is not happy at all about this. So with handover, it is no longer possible to send RTP streams directly between BTS's, but OpenBSC's RTP proxy needs to process the RTP packets and hide the details of the changed source.

This is further complicated by the fact that during handover you are losing speech frames, somewhere between 10 and 40 in the cases that I've seen. This means that when sending the new RTP frames from B*, the sequence number and timestamp needs to account for those lost frames, i.e. incremented by the respective loss count. Otherwise the RTP receiver in BTS A will think it is only receiving old frames and will discard all of them.

Luckily, all of this seems to have been sorted out now and I'm confident we will have actual full handover at the GSM network at 26th annual Chaos Communication Congress in a few days from now. We'll be running 3 BTS's with a total number of 5 TRX's inside the conference building.

German National Education and Research Network reports on OpenBTS and OpenBSC

Harald Welte — Thu, 10 Dec 2009 19:00:00 GMT

Issue 77 of the regular publication "DFN Mitteilungen" of the German National Research Network (DFN) reports on Open Source software for GSM networks, specifically OpenBTS and OpenBSC.

I'm happy to see that at least some parts in academia are now discovering this software and use it for research purpose. That's great news!