LaForge's home page (Posts about osmocom)

OsmoDevCon 2024: "Introduction to XDP, eBPF and AF_XDP"

Harald Welte — Sat, 04 May 2024 16:00:00 GMT

I've presented a talk Introduction to XDP, eBPF and AF_XDP as part of the OsmoDevCon 2024 conference on Open Source Mobile Communications.

This talk provides a generic introduction to a set of modern Linux kernel technologies:

eBPF (extended Berkeley Packet Filter) is a kind of virtual machine that runs sandboxed programs inside the Linux kernel.
XDP (eXpress Data Path) is a framework for eBPF that enables high-performance programmable packet processing in the Linux kernel
AF_XDP is an address family that is optimized for high-performance packet processing. It allows in-kernel XDP eBPF programs to efficiently pass packets to userspace via memory-mapped ring buffers.

The talk provides a high-level overview. It should provide some basics before the other/later talks on bpftrace and eUPF.

You can find the video recording at https://media.ccc.de/v/osmodevcon2024-204-introduction-to-xdp-ebpf-and-afxdp

OsmoDevCon 2024: "Using bpftrace to analyze osmocom performance"

Harald Welte — Sat, 04 May 2024 16:00:00 GMT

I've presented a talk Using bpftrace to analyze osmocom performance as part of the OsmoDevCon 2024 conference on Open Source Mobile Communications.

bpftrace is a utility that uses the Linux kernel tracing infrastructure (and eBPF) in order to provide tracing capabilities within the kernel, like uprobe, kprobe, tracepoints, etc.

bpftrace can help us to analyze the performance of [unmodified] Osmocom programs and quickly provide information like, for example:

Histogram of time spent in a specific system call
Histogram of any argument or return value of any system call

You can find the video recording at https://media.ccc.de/v/osmodevcon2024-203-using-bpftrace-to-analyze-osmocom-performance

OsmoDevCon 2024: "Anatomy of the eSIM Profile"

Harald Welte — Fri, 03 May 2024 16:00:00 GMT

I've presented a talk Anatomy of the eSIM Profile as part of the OsmoDevCon 2024 conference on Open Source Mobile Communications.

In the eSIM universe, eSIM profiles are the virtualised content of a classic USIM (possibly with ISIM, CSIM, applets, etc.).

Let's have a look what an eSIM profile is:

how is the data structured / organized?
what data can be represented in it?
how to handle features provided by eUICC, how can the eSIM profile mandate some of them?
how does personalization of eSIM profiles work?

There is also hands-on navigation through profiles, based on the pySim.esim.saip module.

You can find the video recording at https://media.ccc.de/v/osmodevcon2024-174-anatomy-of-the-esim-profile

OsmoDevCon 2024: "Detailed workings of OTA for SIM/USIM/eUICC"

Harald Welte — Fri, 03 May 2024 16:00:00 GMT

I've presented a talk Detailed workings of OTA for SIM/USIM/eUICC as part of the OsmoDevCon 2024 conference on Open Source Mobile Communications.

Everyone knows that OTA (over the air) access to SIM cards exists for decades, and that somehow authenticated APDUs can be sent via SMS.

But let's look at the OTA architecture in more detail:

OTA transport (SCP80) over SMS, USSD, CellBroadcast, CAT-TP, BIP
The new SCP81 transport (HTTPS via TLS-PSK)
how to address individal applications on the card via their TAR
common applications like RFM and RAM
custom applications on the card
OTA in the world of eUICCs
talking to the ECASD
talking to the ISD-R
talking to the ISD-P/MNO-SD or applications therein

You can find the video recording at https://media.ccc.de/v/osmodevcon2024-175-detailed-workings-of-ota-for-sim-usim-euicc

OsmoDevCon 2024: "GlobalPlatform in USIM and eUICC"

Harald Welte — Fri, 03 May 2024 16:00:00 GMT

I've presented a talk GlobalPlatform in USIM and eUICC as part of the OsmoDevCon 2024 conference on Open Source Mobile Communications.

The GlobalPlatform Card Specification and its many amendments play a significant role in most real-world USIM/ISIM, and even more so in eUICC.

The talk will try to provide an overview of what GlobalPlatform does in the telecommunications context.

Topics include:

security domains
key loading
card and application life cycle
loading and installation of applications
Secure Channel Protocols SCP02, SCP03

You can find the video recording at https://media.ccc.de/v/osmodevcon2024-173-globalplatform-in-usim-and-euicc

OsmoDevCon 2024: "High-performance I/O using io_uring via osmo_io"

Harald Welte — Thu, 02 May 2024 16:00:00 GMT

I've co-presented a talk (together with Andreas Eversberg High-performance I/O using io_uring via osmo_io as part of the OsmoDevCon 2024 conference on Open Source Mobile Communications.

Traditional socket I/O via read/write/recvfrom/sendto/recvmsg/sendmsg and friends creates a very high system call load. A highly-loaded osmo-bsc spends most of its time in syscall entry and syscall exit.

io_uring is a modern Linux kernel mechanism to avoid this syscall overhead. We have introduced the osmo_io`API to libosmocore as a generic back-end for non-blocking/asynchronous I/O and a back-end for our classic `osmo_fd / poll approach as well as a new backend for io_uring.

The talk will cover

a very basic io_uring introduction
a description of the osmo_io API
the difficulties porting from osmo_fd to osmo_io
status of porting various sub-systems over to osmo_io

You can find the video recording at https://media.ccc.de/v/osmodevcon2024-209-high-performance-i-o-using-iouring-via-osmoio

OsmoDevCall: "Osmocom SIMtrace2 Tutorial"

Harald Welte — Tue, 18 Oct 2022 16:00:00 GMT

I've presented an SIMtrace2 Tutorial - SIM protocol tracing: how and why as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20221019-laforge-simtrace2-tutorial

Deployment of future community TDMoIP hub

Harald Welte — Sun, 18 Sep 2022 16:00:00 GMT

I've mentioned some of my various retronetworking projects in some past blog posts. One of those projects is Osmocom Community TDM over IP (OCTOI). During the past 5 or so months, we have been using a number of GPS-synchronized open source icE1usb interconnected by a new, efficient but strill transparent TDMoIP protocol in order to run a distributed TDM/PDH network. This network is currently only used to provide ISDN services to retronetworking enthusiasts, but other uses like frame relay have also been validated.

So far, the central hub of this OCTOI network has been operating in the basement of my home, behind a consumer-grade DOCSIS cable modem connection. Given that TDMoIP is relatively sensitive to packet loss, this has been sub-optimal.

Luckily some of my old friends at noris.net have agreed to host a new OCTOI hub free of charge in one of their ultra-reliable co-location data centres. I'm already hosting some other machines there for 20+ years, and noris.net is a good fit given that they were - in their early days as an ISP - the driving force in the early 90s behind one of the Linux kernel ISDN stracks called u-isdn. So after many decades, ISDN returns to them in a very different way.

Side note: In case you're curious, a reconstructed partial release history of the u-isdn code can be found on gitea.osmocom.org

But I digress. So today, there was the installation of this new OCTOI hub setup. It has been prepared for several weeks in advance, and the hub contains two circuit boards designed entirely only for this use case. The most difficult challenge was the fact that this data centre has no existing GPS RF distribution, and the roof is ~ 100m of CAT5 cable (no fiber!) away from the roof. So we faced the challenge of passing the 1PPS (1 pulse per second) signal reliably through several steps of lightning/over-voltage protection into the icE1usb whose internal GPS-DO serves as a grandmaster clock for the TDM network.

The equipment deployed in this installation currently contains:

a rather beefy Supermicro 2U server with EPYC 7113P CPU and 4x PCIe, two of which are populated with Digium TE820 cards resulting in a total of 16 E1 ports
an icE1usb with RS422 interface board connected via 100m RS422 to an Ericsson GPS03 receiver. There's two layers of of over-voltage protection on the RS422 (each with gas discharge tubes and TVS) and two stages of over-voltage protection in the coaxial cable between antenna and GPS receiver.
a Livingston Portmaster3 RAS server
a Cisco AS5400 RAS server

For more details, see this wiki page and this ticket

Now that the physical deployment has been made, the next steps will be to migrate all the TDMoIP links from the existing user base over to the new hub. We hope the reliability and performance will be much better than behind DOCSIS.

In any case, this new setup for sure has a lot of capacity to connect many more more users to this network. At this point we can still only offer E1 PRI interfaces. I expect that at some point during the coming winter the project for remote TDMoIP BRI (S/T, S0-Bus) connectivity will become available.

Acknowledgements

I'd like to thank anyone helping this effort, specifically * Sylvain "tnt" Munaut for his work on the RS422 interface board (+ gateware/firmware) * noris.net for sponsoring the co-location * sysmocom for sponsoring the EPYC server hardware

Clock sync trouble with Digium cards and timing cables

Harald Welte — Thu, 08 Sep 2022 16:00:00 GMT

If you have ever worked with Digium (now part of Sangoma) digital telephony interface cards such as the TE110/410/420/820 (single to octal E1/T1/J1 PRI cards), you will probably have seen that they always have a timing connector, where the timing information can be passed from one card to another.

In PDH/ISDN (or even SDH) networks, it is very important to have a synchronized clock across the network. If the clocks are drifting, there will be underruns or overruns, with associated phase jumps that are particularly dangerous when analog modem calls are transported.

In traditional ISDN use cases, the clock is always provided by the network operator, and any customer/user side equipment is expected to synchronize to that clock.

So this Digium timing cable is needed in applications where you have more PRI lines than possible with one card, but only a subset of your lines (spans) are connected to the public operator. The timing cable should make sure that the clock received on one port from the public operator should be used as transmit bit-clock on all of the other ports, no matter on which card.

Unfortunately this decades-old Digium timing cable approach seems to suffer from some problems.

bursty bit clock changes until link is up

The first problem is that downstream port transmit bit clock was jumping around in bursts every two or so seconds. You can see an oscillogram of the E1 master signal (yellow) received by one TE820 card and the transmit of the slave ports on the other card at https://people.osmocom.org/laforge/photos/te820_timingcable_problem.mp4

As you can see, for some seconds the two clocks seem to be in perfect lock/sync, but in between there are periods of immense clock drift.

What I'd have expected is the behavior that can be seen at https://people.osmocom.org/laforge/photos/te820_notimingcable_loopback.mp4 - which shows a similar setup but without the use of a timing cable: Both the master clock input and the clock output were connected on the same TE820 card.

As I found out much later, this problem only occurs until any of the downstream/slave ports is fully OK/GREEN.

This is surprising, as any other E1 equipment I've seen always transmits at a constant bit clock irrespective whether there's any signal in the opposite direction, and irrespective of whether any other ports are up/aligned or not.

But ok, once you adjust your expectations to this Digium peculiarity, you can actually proceed.

clock drift between master and slave cards

Once any of the spans of a slave card on the timing bus are fully aligned, the transmit bit clocks of all of its ports appear to be in sync/lock - yay - but unfortunately only at the very first glance.

When looking at it for more than a few seconds, one can see a slow, continuous drift of the slave bit clocks compared to the master :(

Some initial measurements show that the clock of the slave card of the timing cable is drifting at about 12.5 ppb (parts per billion) when compared against the master clock reference.

This is rather disappointing, given that the whole point of a timing cable is to ensure you have one reference clock with all signals locked to it.

The work-around

If you are willing to sacrifice one port (span) of each card, you can work around that slow-clock-drift issue by connecting an external loopback cable. So the master card is configured to use the clock provided by the upstream provider. Its other ports (spans) will transmit at the exact recovered clock rate with no drift. You can use any of those ports to provide the clock reference to a port on the slave card using an external loopback cable.

In this setup, your slave card[s] will have perfect bit clock sync/lock.

Its just rather sad that you need to sacrifice ports just for achieving proper clock sync - something that the timing connectors and cables claim to do, but in reality don't achieve, at least not in my setup with the most modern and high-end octal-port PCIe cards (TE820).

Progress on the ITU-T V5 access network front

Harald Welte — Thu, 08 Sep 2022 16:00:00 GMT

Almost one year after my post regarding first steps towards a V5 implementation, some friends and I were finally able to visit Wobcom, a small German city carrier and pick up a lot of decommissioned POTS/ISDN/PDH/SDH equipment, primarily V5 access networks.

This means that a number of retronetworking enthusiasts now have a chance to play with Siemens Fastlink, Nokia EKSOS and DeTeWe ALIAN access networks/multiplexers.

My primary interest is in Nokia EKSOS, which looks like an rather easy, low-complexity target. As one of the first steps, I took PCB photographs of the various modules/cards in the shelf, take note of the main chip designations and started to search for the related data sheets.

The results can be found in the Osmocom retronetworking wiki, with https://osmocom.org/projects/retronetworking/wiki/Nokia_EKSOS being the main entry page, and sub-pages about

In short: Unsurprisingly, a lot of Infineon analog and digital ICs for the POTS and ISDN ports, as well as a number of Motorola M68k based QUICC32 microprocessors and several unknown ASICs.

So with V5 hardware at my disposal, I've slowly re-started my efforts to implement the LE (local exchange) side of the V5 protocol stack, with the goal of eventually being able to interface those V5 AN with the Osmocom Community TDM over IP network. Once that is in place, we should also be able to offer real ISDN Uk0 (BRI) and POTS lines at retrocomputing events or hacker camps in the coming years.

Retronetworking at VCFB 2022

Harald Welte — Thu, 08 Sep 2022 16:00:00 GMT

I'm happy to announce active participation at the Vintage Computing Festival Berlin 2022 in two ways:

Running a retronetworking exhibit on Modem and ISDN dial-up
Giving a talk on the Osmocom Community TDMoIP netwokr

The exhibit will be similar to the exhibit at the retrocomputing village of the last CCC congress (36C3): A digital telephony network with ISDN BRI and POTS lines providing services to a number of laptops with Modems and ISDN terminal adapters.

We plan to demo the following things: * analog modem and ISDN dial-up into BBSs ** text / ANSI interfaces via Telix, Telemate, Terminate ** RIPterm graphical interfaces * analog modem and ISDN dial-up IP/internet * ISDN video telephony

The client computers will be contemporary 486/Pentium machines wit DOS, Windows 3.11 and OS/2.

OsmoDevCall: "Advanced SIM card topics: GlobalPlatform SCP02, OTA, ARA-M, ISIM"

Harald Welte — Thu, 24 Feb 2022 16:00:00 GMT

I've presented on Advanced SIM card topics: GlobalPlatform SCP02, OTA, ARA-M, ISIM as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20220225-laforge-advanced-sim-topics

OsmoDevCall: "Control/User Plane Separation (CUPS) and PFCP"

Harald Welte — Mon, 27 Dec 2021 16:00:00 GMT

I've presented on Control/User Plane Separation (CUPS) and PFCP as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20211125-laforge-cups-pfcp

OsmoDevCall: "E1, TDM, PDH, SDH, Basics"

Harald Welte — Thu, 11 Nov 2021 16:00:00 GMT

I've presented on E1, TDM, PDH, SDH, Basics as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20211112-laforge-tdm

OsmoDevCall: "SIM card profile creation, personalization, production"

Harald Welte — Thu, 11 Nov 2021 16:00:00 GMT

I've presented on SIM card profile creation, personalization, production as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20211022-laforge-sim

First steps towards an ITU-T V5.1 / V5.2 implementation

Harald Welte — Sun, 10 Oct 2021 16:00:00 GMT

As some of you may know, I've been starting to collect "vintage" telecommunications equipment starting from analog modems to ISDN adapters, but also PBXs and even SDH equipment. The goal is to keep this equipment (and related software) alive for demonstration and practical exploration.

Some [incomplete] information can be found at https://osmocom.org/projects/retro-bbs/wiki/

Working with PBXs to simulate the PSTN (ISDN/POTS) network is fine to some extent, but it's of course not the real deal. You only get S0-buses and no actual Uk0 like actual ISDN lines of the late 80ies and 90ies. You have problems with modems not liking the PBX dialtone, etc.

Hence, I've always wanted to get my hand on some more real-world central-office telephone network equipment, and I finally have a source for so-called V5.1/V5.2 access multiplexers. Those are like remote extension boxes for the central office switch (like EWSD or System 12). They aggregate/multiplex a number of analog or ISDN BRI subscriber lines into E1 lines, while not implementing any of the actual call control or ISDN signalling logic. All of that is provided by the actual telephone switch/exchange.

So in order to integrate such access multiplexers in my retronetworking setup, I will have to implement the LE (local exchange) side of the V5.1 and/or V5.2 protocols, as specified in ITU-T G.964 and G.965.

In the limited spare time I have next to my dayjob and various FOSS projects, progress will likely be slow. Nonetheless I started with an implementation now, and I already had a lot of fun learning about more details of those interfaces and their related protocols.

One of the unresolved questions is to what kind of software I would want to integrate once the V5.x part is resolved.

lcr would probably be the most ISDN-native approach, but it is mostly unused and quite EOL.
Asterisk or FreeSWITCH would of course be obvious candidates, but they are all relatively alien to ISDN, and hence not very transparent once you start to do anything but voice calls (e.g. dialup ISDN data calls in various forms).
yate is another potential candidate. It already supports classic SS7 including ISUP, so it would be a good candidate to build an actual ISDN exchange with V5.2 access multiplexers on the customer-facing side (Q.921+Q.931 on it) and SS7/ISUP towards other exchanges.

For now I think yate would be the most promising approach. Time will tell.

The final goal would then be to have a setup [e.g. at a future CCC congress] where we would have SDH add/drop multiplexers in several halls, and V5.x access multiplexers attached to that, connecting analog and ISDN BRI lines from individual participants to a software-defined central exchange. Ideally actually multiple exchanges, so we can show the signaling on the V5.x side, the Q.921/Q.931 side and the SS7/ISUP between the exchanges.

Given that the next CCC congress is not before December 2022, there is a chance to actually implement this before then ;)

OsmoDevCall: "osmo-remsim in practice"

Harald Welte — Thu, 26 Aug 2021 16:00:00 GMT

I've presented on osmo-remsim in practice as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20210827-laforge-osmo-remsim

OsmoDevCall: "GSM-R and how it differs from GSM"

Harald Welte — Thu, 12 Aug 2021 16:00:00 GMT

I've presented on GSM-R and how it differs from GSM as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20210813-laforge-gsm-r

OsmoDevCall: "High-Level intro to IMS, VoLTE & VoWiFi"

Harald Welte — Thu, 22 Jul 2021 16:00:00 GMT

I've presented a talk High-Level intro to IMS, VoLTE & VoWiFi as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20210723-laforge-ims-volte-vowifi

Notfallwarnung im Mobilfunknetz + Cell Broadcast (Teil 2)

Harald Welte — Mon, 19 Jul 2021 16:00:00 GMT

[excuse this German-language post, this is targeted at the current German public discourse]

Ein paar Ergänzungen zu meinem blog-post gestern.

Ich benutzt den generischen Begriff PWS statt SMSCB, weil SMSCB strikt genommen nur im 2G-System existiert, und nur ein Layer ist, der dort für Notfallalarmierung verwendet wird.

Zu Notfallwarn-Apps

Natürlich sind spezielle, nationale Deutsche Katastrophenschutz-Apps auch nützlich! Aber diese sollten allenfalls zusätzlich angeboten werden, nachdem man erstmal die grundlegende Alarmierung konform der relevanten internationalen (und auch EU-)Standards via Cell Broadcast / PWS realisiert. Man sagt ja auch nicht: Nachrichtensendungen braucht man im Radio nicht mehr, weil man die bereits im Fernsehen hat. Man will auf allen verfügbaren Kanälen senden, und zunächst jene mit möglichst universeller Reichweite und klaren technischen Vorteilen benutzen, bevor man dann zusätzlich auch auf anderen Kanälen alarmiert.

Wie sieht PWS für mich als Anwender aus

Hier scheint es größere Missverständnisse zu geben, wie das auf dem Telefon letztlich aussieht. Ist ja auch verständlich, hierzulande sieht man das nie, ausser man ist zufällig in einem Labor/Basttel-Netz z.B. auf einer CCC-Veranstaltung unterwegs, in der das Osmocom-Projekt mal solche Nachrichten versendet hat.

Die PWS (ETWS, CMAS, WEA, KPAS, EU-ALERT, ...) nachrichten werden vom Telefon empfangen, und dann je nach Konfiguration und Priorität behandelt. Für die USA ist im WEA vorgeschrieben, dass Alarme einer bestimmten Prioritatsklasse (z.B. der Presidential Level Alert) immer zwangsweise zur Anzeige gebracht werden und immer mit einem lauten sirenenartigen Alarmton einhergehen. Es ist sogar explizit verboten, dass der Anwender diese Alarme irgendwo ausstellen, stumm schalten o.ä. kann. Insofern spielt es keine Rolle, ob das Telefon gerade Lautlos gestellt ist, oder es nicht gerade unmittelbar bei mir ist.

Bei manchen Geräten werden die Warnungen sogar mittels einer text2speech-Engine laut über den Lautsprecher vorgelesen, nachdem der Alarmton erscheint. Ob das eine regulatorische Anforderung eines der nationalen System ist, weiss ich nicht - ich habe es jedenfalls bereits in manchen Fällen gesehen, als ich mittels Osmocom-Software solche Alarme in privaten Labornetzen versandt habe.

Noch ein paar technische Details

PWS-Nachrichten werden auch dann noch ausgestrahlt, wenn die Zelle ihre Netzanbindung verloren hat. Wenn also z.B. das Glasfaserkabel zum Kernnetz bereits weg ist, aber noch Strom da ist, werden bereits vorher vom CBC (Cell Broadcast Centre) an die Mobilfunkzelle übermittelte Warnungen entsprechend ihrer Gültigkeitsdauer weiter autonom von der Zelle ausgesendet Das ist wieder ein inhärenter technischer Vorteil, der niemals mit einer App erreichbar ist, weil diese erfordert dass das komplette Mobilfunknetz mit allen internen Verbindungen und dem Kernnetz sowie die Internetverbindung vom Netzbetreiber zum Server des App-Anbieters durchgehend funktioniert.
PWS-Nachrichten können zumindest technisch auch von Telefonen empfangen werden, die garnicht im Netz eingebucht sind, oder die keine SIM eingelegt haben. Ob dies in den Standards gefordert wird, und/oder ob dies die jeweilige Telefonsoftware das so umsetzt, weiss ich nicht und müsste man prüfen. Technisch liegt es nahe, ähnlich wie das Absetzen von Notrufen, das ja auch technisch in diesen Fällen möglich ist.

Zu den Kosten

Wenn - wie in der idealen Welt - das Vorhalten von Notfallalarmierung eine Vorgabe bereits zum Zeitpunkt der Lizenzvergabe für Funkfrequenzen gewesen wäre, wäre das alles einfach ganz lautlos von Anfang an immer unterstützt gewesen. Keiner hätte extra Geld investieren müssen, weil diese minimale technische Vorgabe dann ja bereits Teil der Ausschreibungen der Betreiber für den Einkauf ihres Equipments gewesen wäre. Zudem hatten wir ja bereits in der Vergangenheit Cell Brodacast in allen drei Deutschen Netzen, d.h. die Technik war mal [aus ganz andern Gründen] vorhanden aber wurde irgendwann weggespart.

Das jetzt nachträglich einzuführen heisst natürlich, dass es niemand eingeplant hat, und dass jeder beteiligte am Markt sich das vergolden lassen will. Die Hersteller freuen sich in etwa wie "Oh, Ihr wollt jetzt mehr als ihr damals beim Einkauf spezifiziert habt? Schön, dann schreiben wir mal ein Angebot".

Technisch ist das alles ein Klacks. Die komplette Entwicklung aller Bestandteile für PWS in 2G/3G/4G/5G würde ich auf einen niedrigen einmaligen sechsstelligen Betrag schätzen. Und das ist die einmalige Investition in der Entwicklung, welche dann über alle Geräte/Länder/Netze umgebrochen wird. Bei den Milliarden, die in Entwicklung und Anschaffung von Mobilfunktechnik investiert wird, ist das ein Witz.

Die Geräte wie Basisstationen aller relevanten Hersteller unterstützen natürlich von Haus aus PWS. Die bauen für Deutschland ja nicht andere Geräte, als jene, die in UK, NL, RO, US, ... verbaut werden. Der Markt ist international, die gleiche Technik steht überall.

Weil man jetzt zu spät ist, wird das natürlich von allen Seiten ausgenutzt. Jeder Basisstationshersteller wird die Hand aufhalten und sagen, das kostet jetzt pro Zelle X EUR im Jahr zusätzliche Lizenzgebühren. Und die Anbieter der zentralen Komponente CBC werden auch branchenüblich die Hand aufhalten, mit satten jährlichen Lizenzgebühren. Und die Consultants werden auch alle die Hand aufhalten, weil es gibt wieder etwas zu Integrieren, zu testen, ... Das CBC ist keine komplexe Technik. Wenn man das einmalig als Open Source entwickeln lässt, und in allen Netzen einsetzt, bekommt man es quasi zum Nulltarif. Aber das würde ja Voraussetzen, dass man sich wirklich mit der Technik befasst, versteht um welch simple Software es hier geht, und dass man mal andere Wege in der Beschaffung geht, als nur mal eben bei seinen existierenden 3 Lieferanten anzurufen, die sich dann eine goldene Nase verdienen wollen.

In der öffentlichen Diskussion wird von 20-40 Millionen EUR gesprochen. Das sind überzogene Forderungen der Marktteilnehmer, nichts sonst. Aber selbst wenn man der Meinung ist, dass man lieber das Geld zum Fenster hinauswerfen will, statt Open Source Alternativen zu [ver]suchen, dann ist auch diese Größenordnung etwas, dass im Vergleich zu den sonstigen Anschaffungs- und Betriebskosten eines Mobilfunknetzes verschwindend gering ist. Ganz zu schweigen von den Folgekosten im Bereich Bergung/Rettung, Personenschäden, etc. die sich dadurch mittelfristig bei Katastrophen einsparen lassen.

Oder anders betrachtet: Wenn sogar das wirtschaftlich viel schwächere Rumänien sich sowas leisten kann, dann wird es wohl auch die Bundesrepublik Deutschland stemmen können.

Notfallwarnung im Mobilfunknetz + Cell Broadcast

Harald Welte — Sun, 18 Jul 2021 16:00:00 GMT

[excuse this German-language post, this is targeted at the current German public discourse]

In mehrerern Gegenden Deutschlands gab es verheerende Hochwasser, und die Öffentlichkeit diskutiert deshalb mal wieder die gute alte Frage nach dem adäquaten Mittel der Alarmierung der Bevölkerung.

Es ist einfach nur ein gigantisches Trauerspiel, wie sehr die Deutsche Politik und Verwaltung in diesem Punkt inzwischen seit Jahrzehnten sämtliche relevanten Standards verpennt, und dann immer wieder öffentlich durch fachlich falsche und völlig uninformierte Aussagen auffällt.

Das Thema wurde vor dem aktuellen Hochwasser bereits letztes Jahr im Rahmen des sog. WarnTag öffentlich diskutiert. Auch hier von Seiten der öffentlichen Hand ausschliesslich mit falschen Aussagen, wie z.B. dass es bei Cell Broadcast Datenschutzprobleme gibt. Dabei ist Cell Broadcast die einzige Technologie, wo keine Rückmeldung des einzelnen Netzteilnehmers erfolgt, und das Netz nichtmal weiss, wer die Nachricht empfangen hat, und wo dieser Empfang stattgefunden hat. Ganz wie beim UKW-Radio.

Fakt ist, dass alle digitalen Mobilfunkstandards seit GSM/2G, d.h. seit 1991 die Möglichkeit mitbringen, effizient, schnell und datensparsam alle Nutzer (einer bestimmten geographischen Region) mit sogenannten broadcast Nachrichten zu informieren. Diese Technik, in GSM/2G genannt Cell Broacast (oder auch _SMSCB_), unterscheidet sich Grundlegend von allen anderen Kommunikationsformen im Mobilfunknetz, wie Anrufe und herkömmliche SMS (offiziell SMS-PP). Anrufe, SMS und auch mobile Paketdaten (Internet) werden immer für jeden Teilnehmer individuell auf ihm zugewiesenen Funkressourcen übermittelt. Diese Ressourcen sind beschränkt. Es können in keinem Mobilfunknetz der Welt alle Teilnehmer gleichzeitig telefonieren, oder gleichzeitig SMS empfangen.

Stattdessen benutzt Cell Broadcast - wie der Name bereits unmissverständlich klar macht - Einen broadcast, d.h. Rundsendemechanismus. Eine Nachricht wird einmal gesendet, benötigt also nur eine geteilte Ressource auf der Luftschnittstelle, und wird dann von allen Geräten im Empfangsbereich zeitgleich empfangen und dekodiert. Das ist wie UKW-Radio oder klassisches terrestrisches Fernsehen.

Cell Broadcast wurde bereits in den 1990er Jahren von Deutschen Netzbetreibern benutzt. Und zwar nicht für etwas lebensnotwendiges wie die Notfallsignalisierung, sondern für so banale Dinge wie die Liste jener Vorwahlen, zu denen gerade ein vergünstigter "wandernder Ortstarif" Besteht. Ja, sowas gab es mal bei Vodafone. Oder bei O2 wurden über lange Zeit (aus unbekannten Gründen) die GPS-Koordinaten der jeweiligen Basisstation als Cell Broadcast versendet.

In der folgenden (nun fast abgeschalteten) Mobilfunkgeneration 3G wurde Cell Broadcast leicht umbenannt als Service Area Broadcast beibehalten. Schliesslich gibt es ja Länder mit - anders als in Deutschland - funktionierender und kompetenter Regulierung des Telekommunikationsmarktes, und die langjährig bestehenden gesetzlichen Anforderungen solcher Länder zwingen die Netzbetreiber und auch die Ausrüster der Neztbetreiber, neue Mobilfunkstandards so zu entwickeln, dass die gesetzlichen Vorgaben bzgl. der Alarmierung der Bevölkerung im Notfall funktioniert.

Im Rahmen dieser Standardisierung haben eine Reihe von Ländern innerhalb der 3GPP-Standardisierung (zuständig für 2G, 3G, 4G, 5G) sogenannte Public Warning Systems (PWS) standardisiert. Zu diesen gehören z.B. das Japanische ETWAS (Earthquake and Tsunami Warning System), das Koreanische KPAS (Korean Public Alerting System), das US-Amerikanische WEA (Wireless Emergency Alerts, früher bekannt als CMAS) und auch das EU-ALERT mit den nationalen Implementationen NL-ALERT (Niederlande) und UK-ALERT (Großbritannien) sowie RO-ALERT (Rumänien).

Die zahlreichen Studien und Untersuchungen, die zur Gestaltung obiger Systeme und der internationalen Standards im Mobilfunk geführt haben, weisen auch nochmal nach, was sowieso vorher jedem Techniker offensichtlich erscheint: Eine schelle Alarmierung aller Teilnehmer (einer Region) kann nur über einen Broadcast-Mechanismus erfolgen. In Japan war die Zielvorgabe, die Alarmierung in Erdbebenfällen innerhalb von weniger als 4 Sekunden an die gesamte betroffene Bevölkerung zu übertragen. Und das ist mit PWS möglich!

Die relevanten PWS-Standards in 2G/3G/4G/5G bieten jede Menge nützliche Funktionen:

Benachrichtigung in bestimmten geographischen Regionen
Interoperable Schnittstellen, so dass Netzwerkelemente unterschiedlicher Hersteller miteinander kommunizieren
Konfigurierbare Benachrichtigungstexte, nicht nur in der primären Landessprache, sondern auch in mehreren anderen Sprachen, die dann automatisch je nach Spracheinstellung des Telefons wiedergegeben werden
Unterschiedliche Schweregrade von Alarmierungen
Übermittlung nicht nur im Broadcast, sondern auch im Unicast an jeden Teilnehmer, der gerade in einem Telefongespräch ist, und dessen Telefon gerade währenddessen aus technischen Gründen den Broadcast nicht empfangen würde
Unterschied zwischen Wiederholung einer Übertragung ohne Änderung des Inhalts und einer übertragung mit geändertem Inhalt

Es gibt also seit vielen Jahren internationale Standards, wie sämtliche heute eingesetzten Mobilfunktechniken zur schnellen, effizienten und datensparsamen Alarmierung der Bevölkerung eingesetzt werden können.

Es gibt zahlreiche Länder, die diese Systeme seit langem einsetzen. Das US-Amerikanische WEA wurde nach eigenen Angaben seit 2012 bereits mehr als 61.000 Mal benutzt, um Menschen vor Unwetter oder anderen Katastrophen zu warnen.

Sogar innerhalb der EU hat man das EU-ALERT System spezifiziert, welches weitgehend mit dem amerikanischen WEA identisch ist, und auf die gleichen Techniken aufbaut.

Und dann gibt es Länder wie Deutschland, die es seit genauso vielen Jahren vermissen lassen, durch Gesetze oder Vorschriften

die Netzbetreiber zum Betrieb dieser Broadcast-Technologien in ihrem Netz verpflichtet
die Netzbetreiber zur Bereitstellung von standardisierten Schnittstellen gegenüber den Behörden wie Zivilschutz / Katastrophenschutz zu verpflichten, so das diese selbständig über alle Netzbetreiber Warnungen versenden können
die Gerätehersteller z.B. über Vorschriften des FTEG (Gesetz über Funkanlagen und Telekommunikationsendeinrichtungen) zu Verpflichten, die PWS-Nachrichten anzuzeigen

In den USA, dem vermeintlich viel mehr dem Freien Markt und dem Kapitalismus anhängenden System ist all dies der Regulierungsbehörde FCC möglich. In Deutschland mit seiner sozialen Marktwirtschaft ist es anscheinend unmöglich, den Markt entsprechend zu regulieren. Eine solche Regulierung schafft man in Deutschland nur für wirklich wichtige Themen wie zur Durchsetzung der Bereitstellung von Schnittstellen für die Telekommunikationsüberwachung. Bei so irrelevanten Themen wie dem Katastrophenschutz und der Alarmierung der Bevölkerung braucht man den Markt nicht zu regulieren. Wenn die Netzbetreiber kein PWS anbieten wollen, dann ist das einfach so Gottgegeben, und man kann da ja nichts machen.

Falls jemand sich SMSCB und PWS technisch näher ansehen will: In 2019 haben wir im Osmocom-Projekt eine Open Source Implementation des kompletten Systems von BTS über BSC bis zum CBC, sowie der dazwischen befindlichen Protokolle wie CBSP vorgenommen. Dies wurde freundlicherweise durch den Prototype Fund mit EUR 35k finanziert. Ja, so günstig kann man die nötige Technik zumindest für eine einzelne Mobilfunkgeneration entwickeln...

Man kann also in einem selbst betriebenen Labor-Mobilfunknetz, welches auf Open Source Software basiert mehr in Punkt standardkonformer Notfallalarmierung, als die Deutsche Politik, Verwaltung und Netzbetreiber zusammen hinbekommen.

Wir haben in Deutschland Leute, die diese Standards in und auswendig kennen, sogar daran mitgearbeitet haben. Wir haben Entwickler, die diese Standards implementiert haben. Aber wir schaffen es nicht, das auch mal selbst praktisch zu benutzen - das überlassen wir lieber den anderen Ländern. Wir lassen lieber zuerst die ganze Katastrophenalarmierung mittels Sirenen vergammeln, machen den Netzbetreibern keine Vorgaben, entwicklen komische Apps, die Anwender extra installieren müssen, die prinzipbedingt nicht skalieren und beim Test (WarnTag) nicht funktionieren.

Was für eine Glanzleistung für den hochentwickelten Techhologie-Standort Deutschland.

OsmoDevCall: "SS7 and SIGTRAN in 2G/3G networks"

Harald Welte — Sun, 23 May 2021 16:00:00 GMT

I've presented a talk SS7 and SIGTRAN in 2G/3G networks as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20210514-laforge-ss7-sigtran

OsmoDevCall: "pySim-shell - next generation SIM configuration tool"

Harald Welte — Thu, 08 Apr 2021 16:00:00 GMT

I've presented a talk SS7 and SIGTRAN in 2G/3G networks as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20210409-laforge-pysim-shell

OsmoDevCall: "Osmocom 2020 review"

Harald Welte — Thu, 11 Mar 2021 16:00:00 GMT

I've presented a talk Osmocom 2020 review as part of the OsmoDevCall talk series on Osmocom and related technology.

You can find the video recording at https://media.ccc.de/v/osmodevcall-20210312-laforge-overview-of-2020

36C3 Talks on SIM card technology / Mitel DECT

Harald Welte — Sat, 04 Jan 2020 16:00:00 GMT

At 36C3 in December 2019 I had the pleasure of presenting: One full talk about SIM card technology from A to Z and another talk where I presented together with eventphone team members about Security issues in the Mitel SIP-DECT system.

The SIM card talk was surprisingly successful, both in terms of a full audience on-site, as well as in terms of the number of viewers of the recordings on media.ccc.de. SIM cards are a rather niche topic in the wider IT industry, and my talk was not covering any vulnerabilities or the like. Also, there was nothing novel in the talk: SIM cards have been around for decades, and not much has changed (except maybe eSIM and TLS) in recent years.

In any case, I'm of course happy that it was well received. So far I've received lots of positive feedback.

As I'm working [more than] full time in cellular technology for almost 15 years now, it's sometimes hard to imagine what kind of topics people might be interested in. If you have some kind of suggestion on what kind of subject within my area of expertise you'd like me to talk about, please don't hesitate to reach out.

The Mitel DECT talk also went quite well. I covered about 10 minutes of technical details regarding the reverse engineering of the firmware and the communication protocols of the device. Thanks again to Dieter Spaar for helping with that. He is and remains the best reverse engineer I have met, and it's always a privilege to collaborate on any project. It was of course also nice to see what kind of useful (and/or fun) things the eventphone team have built on top of the knowledge that was gained by protocol-level reverse engineering.

If you want to know more low-level technical detail than the 36C3 talk, I recommend my earlier talk at the OsmoDevCon 2019 about Aastra/Mitel DET base station dissection.

If only I had more time, I would love to work on improving the lack of Free / Open Source Software realted to the DECT protocol family. There's the abandoned deDECTed.org, and the equally abandoned dect.osmocom.org project. The former only deals with the loewst levels of DECT (PHY/MAC). The latter is to a large extent implemented as part of an ancient version of the Linux kernel (I would say this should all run in userspace, like we run all of GSM/UMTS/LTE in userspace today).

If anyone wants to help out, I still think working on the DECT DLC and NWK dissectors for wireshark is the best way to start. It will create a tool that's important for anyone working with the DECT protocols, and it will be more or less a requirement for development and debugging should anyone ever go further in terms of implementing those protocols on either the PP or FP side. You can find my humble beginnings of the related dissectors in the laforge/dect branch of osmocom.org/wireshark.git.

Retronetworking / BBS-Revival setup at #36C3

Harald Welte — Sat, 04 Jan 2020 16:00:00 GMT

After many years of being involved in various projects at the annual Chaos Communication Congress (starting from the audio/vidoe recording team at 15C3), I've finally also departed the GSM team, i.e. the people who operate (Osmocom based) cellular networks at CCC events.

The CCC Camp in August 2019 was slightly different: Instead of helping an Osmocom based 2G/3G network, I decided to put up a nextepc-based LTE network and make that use the 2G/3G HLR (osmo-hlr) via a newly-written DIAMETER-to-GSUP proxy. After lots of hacking on that proxy and fixing various bugs in nextepc (see my laforge/cccamp2019 branch here) this was working rather fine.

For 36C3 in December 2019 I had something different in mind: It was supposed to be the first actual demo of the retronetworking / bbs-revival setup I've been working on during past months. This setup in turn is sort-of a continuation of my talk at 34C3 two years ago: BBSs and early Intenet access in the 1990ies.

Rather than just talking about it, I wanted to be able to show people the real thing: Actual client PCs running (mainly) DOS, dialling over analog modems and phone lines as well as ISDN-TAs and ISDN lines into BBSs, together with early Interent access using SLIP and PPP over the same dial-up lines.

The actual setup can be seen at the Dialup Network In A Box wiki page, together with the 36C3 specific wiki page.

What took most of the time was - interestingly - mainly two topics:

A 1U rack-mount system with four E1 ports. I had lots of old Sangoma Quad-E1 cards in PCI form-factor available, but wanted to use a PC with a more modern/faster CPU than those old first-generation Atom boxes that still had actual PCI slots. Those new mainboards don't have PCI but PCIe. There are plenty of PCIe to PCI bridges and associated products on the market, which worked fine with virtually any PCI card I could find, but not with the Sangoma AFT PCI cards I wanted to use. Seconds to minutes after boot, the PCI-PCIe bridges would always forget their secondary bus number. I suspected excessive power consumption or glitches, but couldn't find anything wrong when looking at the power rails with a scope. Adding additional capacitors on every rail also didn't change it. The !RESET line is also clean. It remains a mystery. I then finally decided to but a new (expensive) DAHDI 4-port E1 PCIe card to move ahead. What a waste of money if you have tons of other E1 cards around.
Various trouble with FreeSWITCH. All I wanted/needed was some simple emulation of a PSTN/ISDN switch, operating in NT mode towards both the Livingston Portmaster 3 RAS and the Auerswald PBX. I would have used lcr, but it supports neither DAHDI nor Sangoma, but only mISDN - and there are no mISDN cards with four E1 ports :( So I decided to go for FreeSWITCH, knowing it has had a long history of ISDN/PRI/E1 support. However, it was a big disappointment. First, there were some segfaults due to a classic pointer deref before NULL-check. Next, libpri and FreeSWITCH have a different idea how channel (timeslot) numbers are structured, rendering any call attempt to fail. Finally, FreeSWITCH decided to blindly overwrite any bearer capabilities IE with 'speech', even if an ISDN dialup call (unrestricted digital information) was being handled. The FreeSWITCH documentation contains tons of references on channel input/output variables related to that - but it turns out their libpri integration doesn't set any of those, nor use any of them on the outbound side.

Anyway, after a lot more time than expected the setup was operational, and we could establish modem calls as well as ISDN dialup calls between the clients and the Portmaster3. The PM3 in turn then was configured to forward the dialup sessions via telnet to a variety of BBSs around the internet. Some exist still (or again) on the public internet. Some others were explicitly (re)created by 36C3 participants for this very BBS-Revival setup.

My personal favorite was finding ACiD Underworld 2.0, one of the few BBSs out there today who support RIPscrip, a protocol used to render vector graphics, text and even mouse-clickable UI via modem connection to a DOS/EGA client program called RIPterm. So we had one RIPterm installation on Novell DOS7 that was just used for dialling into ACiD Underworld 2.0.

Among other things we also tested interoperability between the 1980ies CCC DIY accoustic coupler "Datenklo" and the Portmaster, and confirmed that Windows 2000 could establish multilink-PPP not only over two B-channels (128 kbps) but also over 3 B-Channels (192).

Running this setup for four days meant 36C3 was a quite different experience than many previous CCC congresses:

I was less stressed as I wasn't involved in operating a service that many people would want to use (GSM).
I got engaged with many more people with whom I would normally not have entered a conversation, as they were watching the exhibits/demos and we got to chat about the technology involved and the 'good old days'.

So all in all, despite the last minute FreeSWITCH-patching, it was a much more relaxing and rewarding experience for me.

Special thanks to

Sylvain "tnt" Munaut for spending a lot of time with me at the retronetworking assembly. The fact that I had an E1 interface around was a good way for him to continue development on his ICE40 based bi-directional E1 wiretap. He also helped with setup and teardown.
miaoski and evanslify for reviving two of their old BBSs from Taiwan so we could use them at this event

The retronetworking setup is intended to operate at many other future events, whether CCC related, Vintage Computing or otherwise. It's relatively small and portable.

I'm very much looking forward to the next incarnations. Until then, I will hopefully have more software configured and operational, including a variety of local BBSs (running in VMs/containers), together with the respective networking (FTN, ZConnect, ...) and point software like CrossPoint.

If you are interested in helping out with this project: I'm very much looking for help. It doesn't matter if you're old and have had BBS experience back in the day, or if you're a younger person who wants to learn about communications history. Any help is appreciated. Please reach out to the bbs-revival@lists.osmocom.org mailing list, or directly to me via e-mail.

Software Freedom Podcast #3 about Free Software mobile phone communication

Harald Welte — Thu, 19 Dec 2019 16:00:00 GMT

Recently I had the pleasure of being part of the 3rd incarnation of a new podcast series by the Free Software Foundation Europe: The Software Freedom Podcast.

In episode 3, Matthias and Bonnie of the FSFE are interviewing me about various high-level topics related to [the lack of] Free Software in cellular telephony, as well as some of the projects that I was involved in (Openmoko, Osmocom).

We've also touched the current mainstream / political debate about Huawei and 5G networks, where my position can only be summarized as: It doesn't matter much in which country the related proprietary software is being developed. What we need is Free / Open Source software that can be publicly audited, and we need a method by which the operator can ensure that a given version of that FOSS software is actually executing on his equipment.

Thanks to the FSFE for covering such underdeveloped areas of Free Software, and to use their podcast to distribute related information and ideas.

Sometimes software development is a struggle

Harald Welte — Sat, 28 Sep 2019 16:00:00 GMT

I'm currently working on the firmware for a new project, an 8-slot smart card reader. I will share more about the architecture and design ideas behind this project soon, but today I'll simply write about how hard it sometimes is to actually get software development done. Seemingly trivial things suddenly take ages. I guess everyone writing code knows this, but today I felt like I had to share this story.

Chapter 1 - Introduction

As I'm quite convinced of test-driven development these days, I don't want to simply write firmware code that can only execute in the target, but I'm actually working on a USB CCID (USb Class for Smart Card readers) stack which is hardware-independent, and which can also run entirely in userspace on a Linux device with USB gadget (device) controller. This way it's much easier to instrument, trace, introspect and test the code base, and tests with actual target board hardware are limited to those functions provided by the board.

So the current architecture for development of the CCID implementation looks like this:

Implement the USB CCID device using FunctionFS (I did this some months ago, and in fact developing this was a similarly much more time consuming task than expected, maybe I find time to expand on that)
Attach this USB gadget to a virtual USB bus + host controller using the Linux kernel dummy_hcd module
Talk to a dumb phoenix style serial SIM card reader attached to a USB UART, which is connected to an actual SIM card (or any smart card, for that matter)

By using a "stupid" UART based smart card reader, I am very close to the target environment on a Cortex-M microcntroller, where I also have to talk to a UART and hence implement all the beauty of ISO 7816-3. Hence, the test / mock / development environment is as close as possible to the target environment.

So I implemented the various bits and pieces and ended up at a point where I wanted to test. And I'm not getting any response from the UART / SIM card at all. I check all my code, add lots of debugging, play around with various RTS / DTR / ... handshake settings (which sometimes control power) - no avail.

In the end, after many hours of trial + error I actually inserted a different SIM card and finally, I got an ATR from the card. In more than 20 years of working with smart cards and SIM cards, this is the first time I've actually seen a SIM card die in front of me, with no response whatsoever from the card.

Chapter 2 - Linux is broken

Anyway, the next step was to get the T=0 protocol of ISO 7816-3 going. Since there is only one I/O line between SIM card and reader for both directions, the protocol is a half-duplex protocol. This is unlike "normal" RS232-style UART communication, where you have a separate Rx and Tx line.

On the hardware side, this is most often implemented by simply connecting both the Rx and Tx line of the UART to the SIM I/O pin. This in turn means that you're always getting an echo back for every byte you write.

One could discard such bytes, but then I'm targeting a microcontroller, which should be running eight cards in parallel, at preferably baud-rates up to ~1 megabit speeds, so having to read and discard all those bytes seems like a big waste of resources.

The obvious solution around that is to disable the receiver inside the UART before you start transmitting, and re-enable it after you're done transmitting. This is typically done rather easily, as most UART registers in hardware provide some way to selectively enable transmitter and/or receiver independently.

But since I'm working in Linux userspace in my development environment: How do I approximate this kind of behavior? At least the older readers of this blog will remember something called the CREAD flag of termios. Clearing that flag will disable the receiver. Back in the 1990ies, I did tons of work with serial ports, and I remembered there was such a flag.

So I implement my userspace UART backend and somehow it simply doesn't want to work. Again of course I assume I must be doing something wrong. I'm using strace, I'm single-stepping through code - no avail.

In the end, it turns out that I've just found a bug in the Linux kernel, one that appears to be there at least ever since the git history of linux-2.6.git started. Almost all USB serial device drivers do not implement CREAD, and there is no sotware fall-back implemented in the core serial (or usb-serial) handling that would discard any received bytes inside the kernel if CREAD is cleared. Interestingly, the non-USB serial drivers for classic UARTs attached to local bus, PCI, ... seem to support it.

The problem would be half as much of a problem if the syscall to clear CREAD would actually fail with an error. But no, it simply returns success but bytes continue to be received from the UART/tty :/

So that's the second big surprise of this weekend...

Chapter 3 - Again a broken card?

So I settle for implementing the 'receive as many characters as you wrote' work-around. Once that is done, I continue to test the code. And what happens? Somehow my state machine (implemented using osmo-fsm, of course) for reading the ATR (code found here) somehow never wants to complete. The last byte of the ATR always is missing. How can that be?

Well, guess what, the second SIM card I used is sending a broken, non-spec compliant ATR where the header indicates 9 historical bytes are present, but then in reality only 8 bytes are sent by the card.

Of course every reader has a timeout at that point, but that timeout was not yet implemented in my code, and I also wasn't expecting to hit that timeout.

So after using yet another SIM card (now a sysmoUSIM-SJS1, not sure why I didn't even start with that one), it suddenly works.

After a weekend of detours, each of which I would not have assumed at all before, I finally have code that can obtain the ATR and exchange T=0 TPDUs with cards. Of course I could have had that very easily if I wanted (we do have code in pySim for this, e.g.) but not in the architecture that is as close as it gets to the firmware environment of the microcontroller of my target board.

Fernvale Kits - Lack of Interest - Discount

Harald Welte — Fri, 28 Sep 2018 16:00:00 GMT

Back in December 2014 at 31C3, bunnie and xobs presented about their exciting Fernvale project, how they reverse engineered parts of the MT6260 ARM SoC, which also happens to contain a Mediatek GSM baseband.

Thousands (at least hundreds) of people have seen that talk live. To date, 2506 people (or AIs?) have watched the recordings on youtube, 4859 more people on media.ccc.de.

Given that Fernvale was the closest you could get to having a hackable baseband processor / phone chip, I expected at least as much interest into this project as we received four years earlier with OsmocomBB.

As a result, in early 2015, sysmocom decided to order 50 units of Fernvale DVT2 evaluation kits from bunnie, and to offer them in the sysmocom webshop to ensure the wider community would be able to get the boards they need for research into widely available, inexpensive 2G baseband chips.

This decision was made purely for the perceived benefit of the community: Make an exciting project available for anyone. With that kind of complexity and component density, it's unlikely anyone would ever solder a board themselves. So somebody has to build some and make it available. The mark-up sysmocom put on top of bunnie's manufacturing cost was super minimal, only covering customs/import/shipping fees to Germany, as well as minimal overhead for packing/picking and accounting.

Now it's almost four years after bunnie + xobs' presentation, and of those 50 Fernvale boards, we still have 34 (!) units in stock. That means, only 16 people on this planet ever had an interest in playing with what at the time I thought was one of the most exciting pieces of equipment to play with.

So we lost somewhere on the order of close to 3600 EUR in dead inventory, for something that never was supposed to be a business anyway. That sucks, but I still think it was worth it.

In order to minimize the losses, sysmocom has now discounted the boards and reduced the price from EUR 110 to to EUR 58.82 (excluding VAT). I have very limited hope that this will increase the amount of interest in this project, but well, you got to try :)

In case you're thinking "oh, let's wait some more time, until they hand them out for free", let me tell you: If money is the issue that prevents you from playing with a Fernvale, then please contact me with the details about what you'd want to do with it, and we can see about providing them for free or at substantially reduced cost.

In the worst case, it was ~ 3600 EUR we could have invested in implementing more Osmocom software, which is sad. But would I do it again if I saw a very exciting project? Definitely!

The lesson learned here is probably that even a technically very exciting project backed by world-renowned hackers like bunnie doesn't mean that anyone will actually ever do anything with it, unless they get everything handed on a silver plate, i.e. all the software/reversing work is already done for them by others. And that actually makes me much more sad than the loss of those ~ 3600 EUR in sysmocom's balance sheet.

I also feel even more sorry for bunnie + xobs. They've invested time, money and passion into a project that nobody really seemed to want to get involved and/or take further. ("nobody" is meant figuratively. I know there were/are some enthusiasts who did pick up. I'm talking about the big picture). My condolences to bunnie + xobs!

Wireshark dissector for 3GPP CBSP - traces wanted!

Harald Welte — Tue, 18 Sep 2018 16:00:00 GMT

I recently was reading 3GPP TS 48.049, the specification for the CBSP (Cell Broadcast Service Protocol), which is the protocol between the BSC (Base Station Controller) and the CBC (Cell Broadcast Centre). It is how the CBC according to spec is instructing the BSCs to broadcast the various cell broadcast messages to their respective geographic scope.

While OsmoBTS and OsmoBSC do have support for SMSCB on the CBCH, there is no real interface in OsmoBSC yet on how any external application would instruct it tot send cell broadcasts. The only existing interface is a VTY command, which is nice for testing and development, but hardly a scalable solution.

So I was reading up on the specs, discovered CBSP and thought one good way to get familiar with it is to write a wireshark dissector for it. You can find the result at https://code.wireshark.org/review/#/c/29745/

Now my main problem is that as usual there appear to be no open source implementations of this protocol, so I cannot generate any traces myself. More surprising is that it's not even possible to find any real-world CBSP traces out there. So I'm facing a chicken-and-egg problem. I can only test / verify my wireshark dissector if I find some traces.

So if you happen to have done any work on cell broadcast in 2G network and have a CBSP trace around (or can generate one): Please send it to me, thanks!

Alternatively, you can of course also use the patch linked above, build your own wireshark from scratch, test it and provide feedback. Thanks in either case!

Still alive, just not blogging

Harald Welte — Sun, 26 Aug 2018 16:00:00 GMT

It's been months without any update to this blog, and I feel sad about that. Nothing particular has happened to me, everything is proceeding as usual.

At the Osmocom project we've been making great progress on a variety of fronts, including

3GPP LCLS (Local Call, Local Switch)
Inter-BSC hand-over in osmo-bsc
load Based hand-over in osmo-bsc
reintroducing SCCPlite compatibility to the new BSC code in osmo-bsc / libosmo-sigtran
finishing the first release of the SIMtrace2 firmware
extending test coverage on all fronts, particularly in our TTCN-3 test suites
tons of fixes to the osmo-bts measurement processing / reporting
higher precision time of arrival reporting in osmo-bts
migrating osmocom.org services to new, faster servers

At sysmocom, next to the Osmocom topics above, we've

made the sysmoQMOD remote SIM firmware much more robust and reliable
after months of delays, finally SIMtrace2 hardware kits are available again
created autoamtic testing of pySim-prog and sysmo-usim-util
extended our osmo-gsm-tester based automatic testing setup to include multi-TRX nanoBTS setups

In terms of other topic,

my wife and I have been to a three week motorbike tour all over the Alps in July
I've done tons of servicing (brake piston fittings, brake tubes, fuel line, fixing rust/paint, replacing clutch cable, choke cable, transmission chain, replacing several rusted/worn-out needle bearings, and much more) on my 22year old BMW F650ST to prepare it for many more yers to come. As some type-specific spare parts (mostly plastic parts) are becoming rarer, it was best to take care of replacements sooner than later
some servicing/repairs to my 19 year old Audi A4 car (which passed German mandatory inspection without any deficiency at the first attempt!)
some servicing of my Yamaha FZ6
repaired my Fairphone 2 by swapping the microphone module (mike was mute)
I've re-vamped a lot of the physical/hardware infrastructure for gnumonks.org and other sites I run, which was triggered by having to move racks

OsmoCon 2018 CfP closes on 2018-05-30

Harald Welte — Thu, 24 May 2018 16:00:00 GMT

One of the difficulties with OsmoCon2017 last year was that almost nobody submitted talks / discussions within the deadline, early enough to allow for proper planning.

This lad to the situation where the sysmocom team had to come up with a schedule/agenda on their own. Later on much after the CfP deadline,people then squeezed in talks, making the overall schedule too full.

It is up to you to avoid this situation again in 2018 at OsmoCon2018 by submitting your talk RIGHT NOW. We will be very strict regarding late submissions. So if you would like to shape the Agenda of OsmoCon 2018, this is your chance. Please use it.

We will have to create a schedule soon, as [almost] nobody will register to a conference unless the schedule is known. If there's not sufficient contribution in terms of CfP response from the wider community, don't complain later that 90% of the talks are from sysmocom team members and only about the Cellular Network Infrastructure topics.

You have been warned. Please make your CfP submission in time at https://pretalx.sysmocom.de/osmocon2018/cfp before the CfP deadline on 2018-05-30 23:59 (Europe/Berlin)

OsmoDevCon 2018 retrospective

Harald Welte — Sun, 29 Apr 2018 16:00:00 GMT

One week ago, the annual Osmocom developer meeting (OsmoDevCon 2018) concluded after four long and intense days with old and new friends (schedule can be seen here).

It was already the 7th incarnation of OsmoDevCon, and I have to say that it's really great to see the core Osmocom community come together every year, to share their work and experience with their fellow hackers.

Ever since the beginning we've had the tradition that we look beyond our own projects. In 2012, David Burgess was presenting on OpenBTS. In 2016, Ismael Gomez presented about srsUE + srsLTE, and this year we've had the pleasure of having Sukchan Kim coming all the way from Korea to talk to us about his nextepc project (a FOSS implementation of the Evolved Packet Core, the 4G core network).

What has also been a regular "entertainment" part in recent years are the field trip reports to various [former] satellite/SIGINT/... sites by Dimitri Stolnikov.

All in all, the event has become at least as much about the people than about technology. It's a community of like-minded people that to some part are still working on joint projects, but often work independently and scratch their own itch - whether open source mobile comms related or not.

After some criticism last year, the so-called "unstructured" part of OsmoDevCon has received more time again this year, allowing for exchange among the participants irrespective of any formal / scheduled talk or discussion topic.

In 2018, with the help of c3voc, for the first time ever, we've recorded most of the presentations on video. The results are still in the process of being cut, but are starting to appear at https://media.ccc.de/c/osmodevcon2018.

If you want to join a future OsmoDevCon in person: Make sure you start contributing to any of the many Osmocom member projects now to become eligible. We need you!

Now the sad part is that it will take one entire year until we'll reconvene. May the Osmocom Developer community live long and prosper. I want to meet you guys for many more years at OsmoDevCon!

There is of course the user-oriented OsmoCon 2018 in October, but that's a much larger event with a different audience.

Nevertheless, I'm very much looking forward to that, too.

The OsmoCon 2018 Call for Participation is still running. Please consider submitting talks if you have anything open source mobile communications related to share!

osmo-fl2k - Using USB-VGA dongles as SDR transmitter

Harald Welte — Sun, 22 Apr 2018 16:00:00 GMT

Yesterday, during OsmoDevCon 2018, Steve Markgraf released osmo-fl2k, a new Osmocom member project which enables the use of FL2000 USB-VGA adapters as ultra-low-cost SDR transmitters.

How does it work?

A major part of any VGA card has always been a rather fast DAC which generates the 8-bit analog values for (each) red, green and blue at the pixel clock. Given that fast DACs were very rare/expensive (and still are to some extent), the idea of (ab)using the VGA DAC to transmit radio has been followed by many earlier, mostly proof-of-concept projects, such as Tempest for Eliza in 2001.

However, with osmo-fl2k, for the first time it was possible to completely disable the horizontal and vertical blanking, resulting in a continuous stream of pixels (samples). Furthermore, as the supported devices have no frame buffer memory, the samples are streamed directly from host RAM.

As most USB-VGA adapters appear to have no low-pass filters on their DAC outputs, it is possible to use any of the harmonics to transmit signals at much higher frequencies than normally possible within the baseband of the (max) 157 Mega-Samples per seconds that can be achieved.

osmo-fl2k and rtl-sdr

Steve is the creator of the earlier, complementary rtl-sdr software, which since 2012 transforms USB DVB adapters into general-purpose SDR receivers.

Today, six years later, it is hard to think of where SDR would be without rtl-sdr. Reducing the entry cost of SDR receivers nearly down to zero has done a lot for democratization of SDR technology.

There is hence a big chance that his osmo-fl2k project will attain a similar popularity. Having a SDR transmitter for as little as USD 5 is an amazing proposition.

free riders

Please keep in mind that Steve has done rtl-sdr just for fun, to scratch his own itch and for the "hack value". He chose to share his work with the wider public, in source code, under a free software license. He's a very humble person, he doesn't need to stand in the limelight.

Many other people since have built a business around rtl-sdr. They have grabbed domains with his project name, etc. They are now earning money based on what he has done and shared selflessly, without ever contributing back to the pioneering developers who brought this to all of us in the first place.

So, do we want to bet if history repeats itself? How long will it take for vendors showing up online advertising the USB VGA dongles as "SDR transmitter", possibly even with a surcharge? How long will it take for them to include Steve's software without giving proper attribution? How long until they will violate the GNU GPL by not providing the complete corresponding source code to derivative versions they create?

If you want to thank Steve for his amazing work

reach out to him personally
contribute to his work, e.g.
help to maintain it
package it for distributions
send patches (via osmocom-sdr mailing list)
register an osmocom.org account and update the wiki with more information

And last, but not least, carry on the spirit of "hack value" and democratization of software defined radio.

Thank you, Steve! After rtl-sdr and osmo-fl2k, it's hard to guess what will come next :)

udtrace - Unix domain socket tracing

Harald Welte — Thu, 29 Mar 2018 16:00:00 GMT

When developing applications that exchange data over sockets, every so often you'd like to analyze exactly what kind of data is exchanged over the socket.

For TCP/UDP/SCTP/DCCP or other IP-based sockets, this is rather easy by means of libpcap and tools like tcpdump, tshark or wireshark. However, for unix domain socket, unfortunately no such general capture/tracing infrastructure exists in the Linux kernel.

Interestingly, even after searching for quite a bit I couldn't find any existing tools for this. This is surprising, as unix domain sockets are used by a variety of programs, from sql servers to bind8 ndc all the way to the systemctl tool to manage systemd.

In absence of any kernel support, the two technologies I can think of to implement this is either systemtap or a LD_PRELOAD wrapper.

However, I couldn't find an example for using either of those two to get traces of unix domain soocket communications.

Ok, so I get to write my own. My first idea hence was to implement something based on top of systemtap, the Linux kernel tracing framework. Unfortunately, systemtap was broken in Debian unstable (which I use for decades) at the time, so I went back to the good old LD_PRELOAD shim library / wrapper approach.

The result is called udtrace and can be found at

git clone https://gitea.osmocom.org/laforge/udtrace

or alternatively via its github mirror.

Below is a copy+paste of its README file. Let's hope this tool is useful to other developers, too:

udtrace - Unix Domain socket tracing

This is a LD_PRELOAD wrapper library which can be used to trace the data sent and/or received via unix domain sockets.

Unlike IP based communication that can be captured/traced with pcap programs like tcpdump or wireshark, there is no similar mechanism available for unix domain sockets.

This LD_PRELOAD library intercepts the C library function calls of dynamically linked programs. It will detect all file descriptors representing unix domain sockets and will then print traces of all data sent/received via the socket.

Usage

Simply build libudtrace.so using the make command, and then start your to-be-traced program with

LD_PRELOAD=libudtrace.os

e.g.

LD_PRELOAD=libudtrace.so systemctl status

which will produce output like this:

>>> UDTRACE: Unix Domain Socket Trace initialized (TITAN support DISABLED)
>>> UDTRACE: Adding FD 4
>>> UDTRACE: connect(4, "/run/dbus/system_bus_socket")
4 sendmsg W 00415554482045585445524e414c20
4 sendmsg W 3331333033303330
4 sendmsg W 0d0a4e45474f54494154455f554e49585f46440d0a424547494e0d0a
[...]

Output Format

Currently, udtrace will produc the following output:

At time a FD for a unix domain socket is created:

>>> UDTRACE: Adding FD 8

At time a FD for a unix domain socket is closed:

>>> UDTRACE: Removing FD 8

At time a FD for a unix domain socket is bound or connected:

>>> UDTRACE: connect(9, "/tmp/mncc")

When data is read from the socket:

9 read R 00040000050000004403000008000000680000001c0300002c03000000000000

When data is written to the socket:

9 write W 00040000050000004403000008000000680000001c0300002c03000000000000

Where

9 is the file dsecriptor on which the event happened
read/write is the name of the syscall, could e.g. also be sendmsg / readv / etc.
R|W is Read / Write (from the process point of view)
followed by a hex-dump of the raw data. Only data successfully written (or read) will be printed, not the entire buffer passed to the syscall. The rationale is to only print data that was actually sent to or received from the socket.

TITAN decoder support

Getting hex-dumps is nice and fine, but normally one wants to have a more detailed decode of the data that is being passed on the socket.

For TCP based protocols, there is wireshark. But most protocols on unix domain sockets don't follow inter-operable / public standards, so even if one was to pass the traces into wireshark somehow, there would be no decoder.

In the Osmocom project, we already had some type definitions and decoders for our protocols written in the TTCN-3 programming language, using Eclipse TITAN. In order to build those decoders fro MNCC and PCUIF, please use

make ENABLE_TITAN=1

when building the code.

Please note that this introduces a run-time dependency to libttcn3-dynamic.so, which is (at least on Debian GNU/Linux) not installed in a default library search path, so you will have to use something like:

LD_LIBRARY_PATH=/usr/lib/titan LD_PRELOAD=libudtrace.so systemctl status

34C3 and its Osmocom GSM/UMTS network

Harald Welte — Sun, 31 Dec 2017 16:00:00 GMT

At the 34th annual Chaos Communication Congress, a team of Osmocom folks continued the many years old tradition of operating an experimental Osmocom based GSM network at the event. Though I've originally started that tradition, I'm not involved in installation and/or operation of that network, all the credits go to Lynxis, neels, tsaitgaist and the larger team of volunteers surrounding them. My involvement was only to answer the occasional technical question and to look at bugs that show up in the software during operation, and if possible fix them on-site.

34C3 marks two significant changes in terms of its cellular network:

the new post-nitb Osmocom stack was used, with OsmoBSC, OsmoMSC and OsmoHLR
both an GSM/GPRS network (on 1800 MHz) was operated ,as well as (for the first time) an UMTS network (in the 850 MHz band)

The good news is: The team did great work building this network from scratch, in a new venue, and without relying on people that have significant experience in network operation. Definitely, the team was considerably larger and more distributed than at the time when I was still running that network.

The bad news is: There was a seemingly endless number of bugs that were discovered while operating this network. Some shortcomings were known before, but the extent and number of bugs uncovered all across the stack was quite devastating to me. Sure, at some point from day 2 onwards we had a network that provided [some level of] service, and as far as I've heard, some ~ 23k calls were switched over it. But that was after more than two days of debugging + bug fixing, and we still saw unexplained behavior and crashes later on.

This is such a big surprise as we have put a lot of effort into testing over the last years. This starts from the osmo-gsm-tester software and continuously running test setup, and continues with the osmo-ttcn3-hacks integration tests that mainly I wrote during the last few months. Both us and some of our users have also (successfully!) performed interoperability testing with other vendors' implementations such as MSCs. And last, but not least, the individual Osmocom developers had been using the new post-NITB stack on their personal machines.

So what does this mean?

I'm sorry about the sub-standard state of the software and the resulting problems we've experienced in the 34C3 network. The extent of problems surprised me (and I presume everyone else involved)
I'm grateful that we've had the opportunity to discover all those bugs, thanks to the GSM team at 34C3, as well as Deutsche Telekom for donating 3 ARFCNs from their spectrum, as well as the German regulatory authority Bundesnetzagentur for providing the experimental license in the 850 MHz spectrum.
We need to have even more focus on automatic testing than we had so far. None of the components should be without exhaustive test coverage on at least the most common transactions, including all their failure modes (such as timeouts, rejects, ...)

My preferred method of integration testing has been by using TTCN-3 and Eclipse TITAN to emulate all the interfaces surrounding a single of the Osmocom programs (like OsmoBSC) and then test both valid and invalid transactions. For the BSC, this means emulating MS+BTS on Abis; emulating MSC on A; emulating the MGW, as well as the CTRL and VTY interfaces.

I currently see the following areas in biggest need of integration testing:

OsmoHLR (which needs a GSUP implementation in TTCN-3, which I've created on the spot at 34C3) where we e.g. discovered that updates to the subscriber via VTY/CTRL would surprisingly not result in an InsertSubscriberData to VLR+SGSN
OsmoMSC, particularly when used with external MNCC handlers, which was so far blocked by the lack of a MNCC implementation in TTCN-3, which I've been working on both on-site and after returning back home.
user plane testing for OsmoMGW and other components. We currently only test the control plane (MGCP), but not the actual user plane e.g. on the RTP side between the elements
UMTS related testing on OsmoHNBGW, OsmoMSC and OsmoSGSN. We currently have no automatic testing at all in these areas.

Even before 34C3 and the above-mentioned experiences, I concluded that for 2018 we will pursue a test-driven development approach for all new features added by the sysmocom team to the Osmocom code base. The experience with the many issues at 34C3 has just confirmed that approach. In parallel, we will have to improve test coverage on the existing code base, as outlined above. The biggest challenge will of course be to convince our paying customers of this approach, but I see very little alternative if we want to ensure production quality of our cellular stack.

So here we come: 2018, The year of testing.

Osmocom Review 2017

Harald Welte — Sun, 31 Dec 2017 16:00:00 GMT

As 2017 has just concluded, let's have a look at the major events and improvements in the Osmocom Cellular Infrastructure projects (i.e. those projects dealing with building protocol stacks and network elements for mobile network infrastructure.

I've prepared a detailed year 2017 summary at the osmocom.org website, but let me write a bit about the most note-worthy topics here.

NITB Split

Once upon a time, we implemented everything needed to operate a GSM network inside a single process called OsmoNITB. Those days are now gone, and we have separate OsmoBSC, OsmoMSC, OsmoHLR, OsmoSTP processes, which use interfaces that are interoperable with non-Osmocom implementations (which is what some of our users require).

This change is certainly the most significant change in the close-to-10-year history of the project. However, we have tried to make it as non-intrusive as possible, by using default point codes and IP addresses which will make the individual processes magically talk to each other if installed on a single machine.

We've also released a OsmoNITB Migration Guide, as well as our usual set of user manuals in order to help our users.

We'll continue to improve the user experience, to re-introduce some of the features lost in the split, such as the ability to attach names to the subscribers.

Testing

We have osmo-gsm-tester together with the two physical setups at the sysmocom office, which continuously run the latest Osmocom components and test an entire matrix of different BTSs, software configurations and modems. However, this tests at super low load, and it tests only signalling so far, not user plane yet. Hence, coverage is limited.

We also have unit tests as part of the 'make check' process, Jenkins based build verification before merging any patches, as well as integration tests for some of the network elements in TTCN-3. This is much more than we had until 2016, but still by far not enough, as we had just seen at the fall-out from the sub-optimal 34C3 event network.

OsmoCon

2017 also marks the year where we've for the first time organized a user-oriented event. It was a huge success, and we will for sure have another OsmoCon incarnation in 2018 (most likely in May or June). It will not be back-to-back with the developer conference OsmoDevCon this time.

SIGTRAN stack

We have a new SIGTRAN stakc with SUA, M3UA and SCCP as well as OsmoSTP. This has been lacking a long time.

OsmoGGSN

We have converted OpenGGSN into a true member of the Osmocom family, thereby deprecating OpenGGSN which we had earlier adopted and maintained.

Obtaining the local IP address of an unbound UDP socket

Harald Welte — Thu, 19 Oct 2017 16:00:00 GMT

Sometimes one is finding an interesting problem and is surprised that there is not a multitude of blog post, stackoverflow answers or the like about it.

A (I think) not so uncommon problem when working with datagram sockets is that you may want to know the local IP address that the OS/kernel chooses when sending a packet to a given destination.

In an unbound UDP socket, you basically send and receive packets with any number of peers from a single socket. When sending a packet to destination Y, you simply pass the destination address/port into the sendto() socket function, and the OS/kernel will figure out which of its local IP addresses will be used for reaching this particular destination.

If you're a dumb host with a single default router, then the answer to that question is simple. But in any reasonably non-trivial use case, your host will have a variety of physical and/or virtual network devices with any number of addresses on them.

Why would you want to know that address? Because maybe you need to encode that address as part of a packet payload. In the current use case that we have, it is the OsmoMGW, implementing the IETF MGCP Media Gateway Control Protocol.

So what can you do? You can actually create a new "trial" socket, not bind it to any specific local address/port, but connect() it to the destination of your IP packets. Then you do a getsockname(), which will give you the local address/port the kernel has selected for this socket. And that's exactly the answer to your question. You can now close the "trial" socket and have learned which local IP address the kernel would use if you were to send a packet to that destination.

At least on Linux, this works. While getsockname() is standard BSD sockets API, I'm not sure how portable it is to use it on a socket that has not been explicitly bound by a prior call to bind().

Invited keynote + TTCN-3 talk at netdevconf 2.2 in Seoul

Harald Welte — Mon, 09 Oct 2017 16:00:00 GMT

It was a big surprise that I've recently been invited to give a keynote on netfilter history at netdevconf 2.2.

First of all, I wouldn't have expected netfilter to be that relevant next to all the other [core] networking topics at netdevconf. Secondly, I've not been doing any work on netfilter for about a decade now, so my memory is a bit rusty by now ;)

Speaking of Rusty: Timing wise there is apparently a nice coincidence that I'll be able to meet up with him in Berlin later this month, i.e. hopefully we can spend some time reminiscing about old times and see what kind of useful input he has for the keynote.

I'm also asking my former colleagues and successors in the netfilter project to share with me any note-worthy events or anecdotes, particularly also covering the time after my retirement from the core team. So if you have something that you believe shouldn't miss in a keynote on netfilter project history: Please reach out to me by e-mail ASAP and let me know about it.

To try to fend off the elder[ly] statesmen image that goes along with being invited to give keynotes about the history of projects you were working on a long time ago, I also submitted an actual technical talk: TTCN-3 and Eclipse Titan for testing protocol stacks, in which I'll cover my recent journey into TTCN-3 and TITAN land, and how I think those tools can help us in the Linux [kernel] networking community to productively produce tests for the various protocols.

As usual for netdevconf, there are plenty of other exciting talks in the schedule

I'm very much looking forward to both visiting Seoul again, as well as meeting lots of the excellent people involved in the Linux networking subsystems. See ya!

Purism Librem 5 campaign

Harald Welte — Sat, 02 Sep 2017 16:00:00 GMT

There's a new project currently undergoing crowd funding that might be of interest to the former Openmoko community: The Purism Librem 5 campaign.

Similar to Openmoko a decade ago, they are aiming to build a FOSS based smartphone built on GNU/Linux without any proprietary drivers/blobs on the application processor, from bootloader to userspace.

Furthermore (just like Openmoko) the baseband processor is fully isolated, with no shared memory and with the Linux-running application processor being in full control.

They go beyond what we wanted to do at Openmoko in offering hardware kill switches for camera/phone/baseband/bluetooth. During Openmoko days we assumed it is sufficient to simply control all those bits from the trusted Linux domain, but of course once that might be compromised, a physical kill switch provides a completely different level of security.

I wish them all the best, and hope they can leave a better track record than Openmoko. Sure, we sold some thousands of phones, but the company quickly died, and the state of software was far from end-user-ready. I think the primary obstacles/complexities are verification of the hardware design as well as the software stack all the way up to the UI.

The budget of ~ 1.5 million seems extremely tight from my point of view, but then I have no information about how much Puri.sm is able to invest from other sources outside of the campaign.

If you're a FOSS developer with a strong interest in a Free/Open privacy-first smartphone, please note that they have several job openings, from Kernel Developer to OS Developer to UI Developer. I'd love to see some talents at work in that area.

It's a bit of a pity that almost all of the actual technical details are unspecified at this point (except RAM/flash/main-cpu). No details on the cellular modem/chipset used, no details on the camera, neither on the bluetooth chipset, wifi chipset, etc. This might be an indication of the early stage of their plannings. I would have expected that one has ironed out those questions before looking for funding - but then, it's their campaign and they can run it as they see it fit!

I for my part have just put in a pledge for one phone. Let's see what will come of it. In case you feel motivated by this post to join in: Please keep in mind that any crowdfunding campaign bears significant financial risks. So please make sure you made up your mind and don't blame my blog post for luring you into spending money :)

The sad state of voice support in cellular modems

Harald Welte — Fri, 01 Sep 2017 16:00:00 GMT

Cellular modems have existed for decades and come in many shapes and kinds. They contain the cellular baseband processor, RF frontend, protocol stack software and anything else required to communicate with a cellular network. Basically a phone without display or input.

During the last decade or so, the vast majority of cellular modems come as LGA modules, i.e. a small PCB with all components on the top side (and a shielding can), which has contact pads on the bottom so you can solder it onto your mainboard. You can obtain them from vendors such as Sierra Wireless, u-blox, Quectel, ZTE, Huawei, Telit, Gemalto, and many others.

In most cases, the vendors now also solder those modules to small adapter boards to offer the same product in mPCIe form-factor. Other modems are directly manufactured in mPCIe or NGFF aka m.2 form-factor.

As long as those modems were still 2G / 2.5G / 2.75G, the main interconnection with the host (often some embedded system) was a serial UART. The Audio input/output for voice calls was made available as analog signals, ready to connect a microphone and spekaer, as that's what the cellular chipsets were designed for in the smartphones. In the Openmoko phones we also interfaced the audio of the cellular modem in analog, exactly for that reason.

From 3G onwards, the primary interface towards the host is now USB, with the modem running as a USB device. If your laptop contains a cellular modem, you will see it show up in the lsusb output.

From that point onwards, it would have made a lot of sense to simply expose the audio also via USB. Simply offer a multi-function USB device that has both whatever virutal serial ports for AT commands and network device for IP, and add a USB Audio device to it. It would simply show up as a "USB sound card" to the host, with all standard drivers working as expected. Sadly, nobody seems to have implemented this, at least not in a supported production version of their product

Instead, what some modem vendors have implemented as an ugly hack is the transport of 8kHz 16bit PCM samples over one of the UARTs. See for example the Quectel UC-20 or the Simcom SIM7100 which implement such a method.

All the others ignore any acess to the audio stream from software to a large part. One wonders why that is. From a software and systems architecture perspective it would be super easy. Instead, what most vendors do, is to expose a digital PCM interface. This is suboptimal in many ways:

there is no mPCIe standard on which pins PCM should be exposed
no standard product (like laptop, router, ...) with mPCIe slot will have anything connected to those PCM pins

Furthermore, each manufacturer / modem seems to support a different subset of dialect of the PCM interface in terms of

voltage (almost all of them are 1.8V, while mPCIe signals normally are 3.3V logic level)
master/slave (almost all of them insist on being a clock master)
sample format (alaw/ulaw/linear)
clock/bit rate (mostly 2.048 MHz, but can be as low as 128kHz)
frame sync (mostly short frame sync that ends before the first bit of the sample)
endianness (mostly MSB first)
clock phase (mostly change signals at rising edge; sample at falling edge)

It's a real nightmare, when it could be so simple. If they implemented USB-Audio, you could plug a cellular modem into any board with a mPCIe slot and it would simply work. As they don't, you need a specially designed mainboard that implements exactly the specific dialect/version of PCM of the given modem.

By the way, the most "amazing" vendor seems to be u-blox. Their Modems support PCM audio, but only the solder-type version. They simply didn't route those signals to the mPCIe slot, making audio impossible to use when using a connectorized modem. How inconvenient.

Summary

If you want to access the audio signals of a cellular modem from software, then you either

have standard hardware and pick one very specific modem model and hope this is available sufficiently long during your application, or
build your own hardware implementing a PCM slave interface and then pick + choose your cellular modem

On the Osmocom mpcie-breakout board and the sysmocom QMOD board we have exposed the PCM related pins on 2.54mm headers to allow for some separate board to pick up that PCM and offer it to the host system. However, such separate board hasn't been developed so far.

OsmoDevCon 2017 Review

Harald Welte — Tue, 02 May 2017 16:00:00 GMT

After the public user-oriented OsmoCon 2017, we also recently had the 6th incarnation of our annual contributors-only Osmocom Developer Conference: The OsmoDevCon 2017.

This is a much smaller group, typically about 20 people, and is limited to actual developers who have a past record of contributing to any of the many Osmocom projects.

We had a large number of presentation and discussions. In fact, so large that the schedule of talks extended from 10am to midnight on some days. While this is great, it also means that there was definitely too little time for more informal conversations, chatting or even actual work on code.

We also have such a wide range of topics and scope inside Osmocom, that the traditional ad-hoch scheduling approach no longer seems to be working as it used to. Not everyone is interested in (or has time for) all the topics, so we should group them according to their topic/subject on a given day or half-day. This will enable people to attend only those days that are relevant to them, and spend the remaining day in an adjacent room hacking away on code.

It's sad that we only have OsmoDevCon once per year. Maybe that's actually also something to think about. Rather than having 4 days once per year, maybe have two weekends per year.

Always in motion the future is.

Overhyped Docker

Harald Welte — Tue, 02 May 2017 16:00:00 GMT

Overhyped Docker missing the most basic features

I've always been extremely skeptical of suddenly emerging over-hyped technologies, particularly if they advertise to solve problems by adding yet another layer to systems that are already sufficiently complex themselves.

There are of course many issues with containers, ranging from replicated system libraries and the basic underlying statement that you're giving up on the system packet manager to properly deal with dependencies.

I'm also highly skeptical of FOSS projects that are primarily driven by one (VC funded?) company. Especially if their offering includes a so-called cloud service which they can stop to operate at any given point in time, or (more realistically) first get everybody to use and then start charging for.

But well, despite all the bad things I read about it over the years, on one day in May 2017 I finally thought let's give it a try. My problem to solve as a test balloon is fairly simple.

My basic use case

The plan is to start OsmoSTP, the m3ua-testtool and the sua-testtool, which both connect to OsmoSTP. By running this setup inside containers and inside an internal network, we could then execute the entire testsuite e.g. during jenkins test without having IP address or port number conflicts. It could even run multiple times in parallel on one buildhost, verifying different patches as part of the continuous integration setup.

This application is not so complex. All it needs is three containers, an internal network and some connections in between. Should be a piece of cake, right?

But enter the world of buzzword-fueled web-4000.0 software-defined virtualised and orchestrated container NFW + SDN vodoo: It turns out to be impossible, at least not with the preferred tools they advertise.

Dockerfiles

The part that worked relatively easily was writing a few Dockerfiles to build the actual containers. All based on debian:jessie from the library.

As m3ua-testsuite is written in guile, and needs to build some guile plugin/extension, I had to actually include guile-2.0-dev and other packages in the container, making it a bit bloated.

I couldn't immediately find a nice example Dockerfile recipe that would allow me to build stuff from source outside of the container, and then install the resulting binaries into the container. This seems to be a somewhat weak spot, where more support/infrastructure would be helpful. I guess the idea is that you simply install applications via package feeds and apt-get. But I digress.

So after some tinkering, I ended up with three docker containers:

one running OsmoSTP
one running m3ua-testtool
one running sua-testtool

I also managed to create an internal bridged network between the containers, so the containers could talk to one another.

However, I have to manually start each of the containers with ugly long command line arguments, such as docker run --network sigtran --ip 172.18.0.200 -it osmo-stp-master. This is of course sub-optimal, and what Docker Services + Stacks should resolve.

Services + Stacks

The idea seems good: A service defines how a given container is run, and a stack defines multiple containers and their relation to each other. So it should be simple to define a stack with three services, right?

Well, it turns out that it is not. Docker documents that you can configure a static ipv4_address [1] for each service/container, but it seems related configuration statements are simply silently ignored/discarded [2], [3], [4].

This seems to be related that for some strange reason stacks can (at least in later versions of docker) only use overlay type networks, rather than the much simpler bridge networks. And while bridge networks appear to support static IP address allocations, overlay apparently doesn't.

I still have a hard time grasping that something that considers itself a serious product for production use (by a company with estimated value over a billion USD, not by a few hobbyists) that has no support for running containers on static IP addresses. that. How many applications out there have I seen that require static IP address configuration? How much simpler do setups get, if you don't have to rely on things like dynamic DNS updates (or DNS availability at all)?

So I'm stuck with having to manually configure the network between my containers, and manually starting them by clumsy shell scripts, rather than having a proper abstraction for all of that. Well done :/

Exposing Ports

Unrelated to all of the above: If you run some software inside containers, you will pretty soon want to expose some network services from containers. This should also be the most basic task on the planet.

However, it seems that the creators of docker live in the early 1980ies, where only TCP and UDP transport protocols existed. They seem to have missed that by the late 1990ies to early 2000s, protocols like SCTP or DCCP were invented.

But yet, in 2017, Docker chooses to

blindly assume TCP in https://docs.docker.com/engine/reference/builder/#expose without even mentioning it (or designing the syntax to accept any specification of the protocol)
design a syntax (/tcp) in the command-line parsing (see https://docs.docker.com/engine/reference/run/#expose-incoming-ports), but then only parse tcp and udp, despite people requesting support for other protocols like SCTP as early as three years ago

Now some of the readers may think 'who uses SCTP anyway'. I will give you a straight answer: Everyone who has a mobile phone uses SCTP. This is due to the fact that pretty much all the connections inside cellular networks (at least for 3G/4G networks, and in reality also for many 2G networks) are using SCTP as underlying transport protocol, from the radio access network into the core network. So every time you switch your phone on, or do anything with it, you are using SCTP. Not on your phone itself, but by all the systems that form the network that you're using. And with the drive to C-RAN, NFV, SDN and all the other buzzwords also appearing in the Cellular Telecom field, people should actually worry about it, if they want to be a part of the software stack that is used in future cellular telecom systems.

Summary

After spending the better part of a day to do something that seemed like the most basic use case for running three networked containers using Docker, I'm back to step one: Most likely inventing some custom scripts based on unshare to run my three test programs in a separate network namespace for isolated execution of test suite execution as part of a Jenkins CI setup :/

It's also clear that Docker apparently don't care much about playing a role in the Cellular Telecom world, which is increasingly moving away from proprietary and hardware-based systems (like STPs) to virtualised, software-based systems.

OsmoCon 2017 Review

Harald Welte — Sun, 30 Apr 2017 16:00:00 GMT

It's already one week past the event, so I really have to sit down and write some rewview on the first public Osmocom Conference ever: OsmoCon 2017.

The event was a huge success, by all accounts.

We've not only been sold out, but we also had to turn down some last minute registrations due to the venue being beyond capacity (60 seats). People traveled from Japan, India, the US, Mexico and many other places to attend.
We've had an amazing audience ranging from commercial operators to community cellular operators to professional developers doing work relate to osmocom, academia, IT security crowds and last but not least enthusiasts/hobbyists, with whom the project[s] started.
I've received exclusively positive feedback from many attendees
We've had a great programme. Some part of it was of introductory nature and probably not too interesting if you've been in Osmocom for a few years. However, the work on 3G as well as the current roadmap was probably not as widely known yet. Also, I really loved to see Roch's talk about Running a commercial cellular network with Osmocom software as well as the talk on Facebook's OpenCellular BTS hardware and the Community Cellular Manager.
We have very professional live streaming + video recordings courtesy of the C3VOC team. Thanks a lot for your support and for having the video recordings of all talks online already at the next day after the event.

We also received some requests for improvements, many of which we will hopefully consider before the next Osmocom Conference:

have a multiple day event. Particularly if you're traveling long-distance, it is a lot of overhead for a single-day event. We of course fully understand that. On the other hand, it was the first Osmocom Conference, and hence it was a test balloon where it was initially unclear if we'll be able to get a reasonable number of attendees interested at all, or not. And organizing an event with venue and talks for multiple days if in the end only 10 people attend would have been a lot of effort and financial risk. But now that we know there are interested folks, we can definitely think of a multiple day event next time
Signs indicating venue details on the last meters. I agree, this cold have been better. The address of the venue was published, but we could have had some signs/posters at the door pointing you to the right meeting room inside the venue. Sorry for that.
Better internet connectivity. This is a double-edged sword. Of course we want our audience to be primarily focused on the talks and not distracted :P I would hope that most people are able to survive a one day event without good connectivity, but for sure we will have to improve in case of a multiple-day event in the future

In terms of my requests to the attendees, I only have one

Participate in the discussions on the schedule/programme while it is still possible to influence it. When we started to put together the programme, I posted about it on the openbsc mailing list and invited feedback. Still, most people seem to have missed the time window during which talks could have been submitted and the schedule still influenced before finalizing it
Register in time. We have had almost no registrations until about two weeks ahead of the event (and I was considering to cancel it), and then suddenly were sold out in the week ahead of the event. We've had people who first booked their tickets, only to learn that the tickets were sold out. I guess we will introduce early bird pricing and add a very expensive last minute ticket option next year in order to increase motivation to register early and thus give us flexibility regarding venue planning.

Thanks again to everyone involved in OsmoCon 2017!

Ok, now, all of you who missed the event: Go to https://media.ccc.de/c/osmocon17 and check out the recordings. Have fun!

Things you find when using SCTP on Linux

Harald Welte — Sun, 16 Apr 2017 16:00:00 GMT

Observations on SCTP and Linux

When I was still doing Linux kernel work with netfilter/iptables in the early 2000's, I was somebody who actually regularly had a look at the new RFCs that came out. So I saw the SCTP RFCs, SIGTRAN RFCs, SIP and RTP, etc. all released during those years. I was quite happy to see that for new protocols like SCTP and later DCCP, Linux quickly received a mainline implementation.

Now most people won't have used SCTP so far, but it is a protocol used as transport layer in a lot of telecom protocols for more than a decade now. Virtually all protocols that have traditionally been spoken over time-division multiplex E1/T1 links have been migrated over to SCTP based protocol stackings.

Working on various Open Source telecom related projects, i of course come into contact with SCTP every so often. Particularly some years back when implementing the Erlang SIGTAN code in erlang/osmo_ss7 and most recently now with the introduction of libosmo-sigtran with its OsmoSTP, both part of the libosmo-sccp repository.

I've also hard to work with various proprietary telecom equipment over the years. Whether that's some eNodeB hardware from a large brand telecom supplier, or whether it's a MSC of some other vendor. And they all had one thing in common: Nobody seemed to use the Linux kernel SCTP code. They all used proprietary implementations in userspace, using RAW sockets on the kernel interface.

I always found this quite odd, knowing that this is the route that you have to take on proprietary OSs without native SCTP support, such as Windows. But on Linux? Why? Based on rumors, people find the Linux SCTP implementation not mature enough, but hard evidence is hard to come by.

As much as it pains me to say this, the kind of Linux SCTP bugs I have seen within the scope of our work on Osmocom seem to hint that there is at least some truth to this (see e.g. https://bugzilla.redhat.com/show_bug.cgi?id=1308360 or https://bugzilla.redhat.com/show_bug.cgi?id=1308362).

Sure, software always has bugs and will have bugs. But we at Osmocom are 10-15 years "late" with our implementations of higher-layer protocols compared to what the mainstream telecom industry does. So if we find something, and we find it even already during R&D of some userspace code, not even under load or in production, then that seems a bit unsettling.

One would have expected, with all their market power and plenty of Linux-based devices in the telecom sphere, why did none of those large telecom suppliers invest in improving the mainline Linux SCTP code? I mean, they all use UDP and TCP of the kernel, so it works for most of the other network protocols in the kernel, but why not for SCTP? I guess it comes back to the fundamental lack of understanding how open source development works. That it is something that the given industry/user base must invest in jointly.

The leatest discovered bug

During the last months, I have been implementing SCCP, SUA, M3UA and OsmoSTP (A Signal Transfer Point). They were required for an effort to add 3GPP compliant A-over-IP to OsmoBSC and OsmoMSC.

For quite some time I was seeing some erratic behavior when at some point the STP would not receive/process a given message sent by one of the clients (ASPs) connected. I tried to ignore the problem initially until the code matured more and more, but the problems remained.

It became even more obvious when using Michael Tuexen's m3ua-testtool, where sometimes even the most basic test cases consisting of sending + receiving a single pair of messages like ASPUP -> ASPUP_ACK was failing. And when the test case was re-tried, the problem often disappeared.

Also, whenever I tried to observe what was happening by meas of strace, the problem would disappear completely and never re-appear until strace was detached.

Of course, given that I've written several thousands of lines of new code, it was clear to me that the bug must be in my code. Yesterday I was finally prepare to accept that it might actually be a Linux SCTP bug. Not being able to reproduce that problem on a FreeBSD VM also pointed clearly into this direction.

Now I could simply have collected some information and filed a bug report (which some kernel hackers at RedHat have thankfully invited me to do!), but I thought my use case was too complex. You would have to compile a dozen of different Osmocom libraries, configure the STP, run the scheme-language m3ua-testtool in guile, etc. - I guess nobody would have bothered to go that far.

So today I tried to implement a test case that reproduced the problem in plain C, without any external dependencies. And for many hours, I couldn't make the bug to show up. I tried to be as close as possible to what was happening in OsmoSTP: I used non-blocking mode on client and server, used the SCTP_NODELAY socket option, used the sctp_rcvmsg() library wrapper to receive events, but the bug was not reproducible.

Some hours later, it became clear that there was one setsockopt() in OsmoSTP (actually, libosmo-netif) which enabled all existing SCTP events. I did this at the time to make sure OsmoSTP has the maximum insight possible into what's happening on the SCTP transport layer, such as address fail-overs and the like.

As it turned out, adding that setsockopt for SCTP_FLAGS to my test code made the problem reproducible. After playing around which of the flags, it seems that enabling the SENDER_DRY_EVENT flag makes the bug appear.

You can find my detailed report about this issue in https://bugzilla.redhat.com/show_bug.cgi?id=1442784 and a program to reproduce the issue at http://people.osmocom.org/laforge/sctp-nonblock/sctp-dry-event.c

Inside the Osmocom world, luckily we can live without the SENDER_DRY_EVENT and a corresponding work-around has been submitted and merged as https://gerrit.osmocom.org/#/c/2386/

With that work-around in place, suddenly all the m3ua-testtool and sua-testtool test cases are reliably green (PASSED) and OsmoSTP works more smoothly, too.

What do we learn from this?

Free Software in the Telecom sphere is getting too little attention. This is true even those small portions of telecom relevant protocols that ended up in the kernel like SCTP or more recently the GTP module I co-authored. They are getting too little attention in development, even more lack of attention in maintenance, and people seem to focus more on not using it, rather than fixing and maintaining what is there.

It makes me really sad to see this. Telecoms is such a massive industry, with billions upon billions of revenue for the classic telecom equipment vendors. Surely, they would be able to co-invest in some basic infrastructure like proper and reliable testing / continuous integration for SCTP. More recently, we see millions and more millions of VC cash burned by buzzword-flinging companies doing "NFV" and "SDN". But then rather reimplement network stacks in userspace than to fix, complete and test those little telecom infrastructure components which we have so far, like the SCTP protocol :(

Where are the contributions to open source telecom parts from Ericsson, Nokia (former NSN), Huawei and the like? I'm not even dreaming about the actual applications / network elements, but merely the maintenance of something as basic as SCTP. To be fair, Motorola was involved early on in the Linux SCTP code, and Huawei contributed a long series of fixes in 2013/2014. But that's not the kind of long-term maintenance contribution that one would normally expect from the primary interest group in SCTP.

Finally, let me thank to the Linux SCTP maintainers. I'm not complaining about them! They're doing a great job, given the arcane code base and the fact that they are not working for a company that has SCTP based products as their core business. I'm sure the would love more support and contributions from the Telecom world, too.

OsmoCon 2017 Updates: Travel Grants and Schedule

Harald Welte — Sun, 26 Mar 2017 16:00:00 GMT

April 21st is approaching fast, so here some updates. I'm particularly happy that we now have travel grants available. So if the travel expenses were preventing you from attending so far: This excuse is no longer valid!

Get your ticket now, before it is too late. There's a limited number of seats available.

OsmoCon 2017 Schedule

The list of talks for OsmoCon 2017 has been available for quite some weeks, but today we finally published the first actual schedule.

As you can see, the day is fully packed with talks about Osmocom cellular infrastructure projects. We had to cut some talk slots short (30min instead of 45min), but I'm confident that it is good to cover a wider range of topics, while at the same time avoiding fragmenting the audience with multiple tracks.

OsmoCon 2017 Travel Grants

We are happy to announce that we have received donations to permit for providing travel grants!

This means that any attendee who is otherwise not able to cover their travel to OsmoCon 2017 (e.g. because their interest in Osmocom is not related to their work, or because their employer doesn't pay the travel expenses) can now apply for such a travel grant.

For more details see OsmoCon 2017 Travel Grants and/or contact osmocon2017@sysmocom.de.

Upcoming v3 of Open Hardware miniPCIe WWAN modem USB breakout board

Harald Welte — Thu, 23 Mar 2017 16:00:00 GMT

Back in October 2016 I designed a small open hardware breakout board for WWAN modems in mPCIe form-factor. I was thinking some other people might be interested in this, and indeed, the first manufacturing batch is already sold out by now.

Instead of ordering more of the old (v2) design, I decided to do some improvements in the next version:

add mounting holes so the PCB can be mounted via M3 screws
add U.FL and SMA sockets, so the modems are connected via a short U.FL to U.FL cable, and external antennas or other RF components can be attached via SMA. This provides strain relief for the external antenna or cabling and avoids tearing off any of the current loose U.FL to SMA pigtails
flip the SIM slot to the top side of the PCB, so it can be accessed even after mounting the board to some base plate or enclosure via the mounting holes
more meaningful labeling of the silk screen, including the purpose of the jumpers and the input voltage.

A software rendering of the resulting v3 PCB design files that I just sent for production looks like this:

Like before, the design of the board (including schematics and PCB layout design files) is available as open hardware under CC-BY-SA license terms. For more information see http://osmocom.org/projects/mpcie-breakout/wiki

It will take some expected three weeks until I'll see the first assembled boards.

I'm also planning to do a M.2 / NGFF version of it, but haven't found the time to get around doing it so far.

Gory details of USIM authentication sequence numbers

Harald Welte — Tue, 07 Mar 2017 16:00:00 GMT

I always though I understood UMTS AKA (authentication and key agreement), including the re-synchronization procedure. It's been years since I wrote tools like osmo-sim-auth which you can use to perform UMTS AKA with a SIM card inserted into a PC reader, i.e. simulate what happens between the AUC (authentication center) in a network and the USIM card.

However, it is only now as the sysmocom team works on 3G support of the dedicated OsmoHLR (outside of OsmoNITB!), that I seem to understand all the nasty little details.

I always thought for re-synchronization it is sufficient to simply increment the SQN (sequence number). It turns out, it isn't as there is a MSB-portion called SEQ and a lower-bit portion called IND, used for some fancy array indexing scheme of buckets of highest-used-SEQ within that IND bucket.

If you're interested in all the dirty details and associated spec references (the always hide the important parts in some Annex) see the discussion between Neels and me in Osmocom redmine issue 1965.

Manual testing of Linux Kernel GTP module

Harald Welte — Thu, 23 Feb 2017 16:00:00 GMT

In May 2016 we got the GTP-U tunnel encapsulation/decapsulation module developed by Pablo Neira, Andreas Schultz and myself merged into the 4.8.0 mainline kernel.

During the second half of 2016, the code basically stayed untouched. In early 2017, several patch series of (at least) three authors have been published on the netdev mailing list for review and merge.

This poses the very valid question on how do we test those (sometimes quite intrusive) changes. Setting up a complete cellular network with either GPRS/EGPRS or even UMTS/HSPA is possible using OsmoSGSN and related Osmocom components. But it's of course a luxury that not many Linux kernel networking hackers have, as it involves the availability of a supported GSM BTS or UMTS hNodeB. And even if that is available, there's still the issue of having a spectrum license, or a wired setup with coaxial cable.

So as part of the recent discussions on netdev, I tested and described a minimal test setup using libgtpnl, OpenGGSN and sgsnemu.

This setup will start a mobile station + SGSN emulator inside a Linux network namespace, which talks GTP-C to OpenGGSN on the host, as well as GTP-U to the Linux kernel GTP-U implementation.

In case you're interested, feel free to check the following wiki page: https://osmocom.org/projects/linux-kernel-gtp-u/wiki/Basic_Testing

This is of course just for manual testing, and for functional (not performance) testing only. It would be great if somebody would pick up on my recent mail containing some suggestions about an automatic regression testing setup for the kernel GTP-U code. I have way too many spare-time projects in desperate need of some attention to work on this myself. And unfortunately, none of the telecom operators (who are the ones benefiting most from a Free Software accelerated GTP-U implementation) seems to be interested in at least co-funding or otherwise contributing to this effort :/

Osmocom Conference 2017 on April 21st

Harald Welte — Tue, 31 Jan 2017 16:00:00 GMT

I'm very happy that in 2017, we will have the first ever technical conference on the Osmocom cellular infrastructure projects.

For many years, we have had a small, invitation only event by Osmocom developers for Osmocom developers called OsmoDevCon. This was fine for the early years of Osmocom, but during the last few years it became apparent that we also need a public event for our many users. Those range from commercial cellular operators to community based efforts like Rhizomatica, and of course include the many research/lab type users with whom we started.

So now we'll have the public OsmoCon on April 21st, back-to-back with the invitation-only OsmoDevcon from April 22nd through 23rd.

I'm hoping we can bring together a representative sample of our user base at OsmoCon 2017 in April. Looking forward to meet you all. I hope you're also curious to hear more from other users, and of course the development team.

Regards,: Harald

Open Hardware IEEE 802.15.4 adapter "ATUSB" available again

Harald Welte — Tue, 06 Dec 2016 17:00:00 GMT

Many years ago, in the aftermath of Openmoko shutting down, fellow former Linux kernel hacker Werner Almesberger was working on an IEEE 802.15.4 (WPAN) adapter for the Ben Nanonote.

As a spin-off to that, the ATUSB device was designed: A general-purpose open hardware (and FOSS firmware + driver) IEEE 802.15.4 adapter that can be plugged into any USB port.

This adapter has received a mainline Linux kernel driver written by Werner Almesberger and Stefan Schmidt, which was eventually merged into mainline Linux in May 2015 (kernel v4.2 and later).

Earlier in 2016, Stefan Schmidt (the current ATUSB Linux driver maintainer) approached me about the situation that ATUSB hardware was frequently asked for, but currently unavailable in its physical/manufactured form. As we run a shop with smaller electronics items for the wider Osmocom community at sysmocom, and we also frequently deal with contract manufacturers for low-volume electronics like the SIMtrace device anyway, it was easy to say "yes, we'll do it".

As a result, ready-built, programmed and tested ATUSB devices are now finally available from the sysmocom webshop

Note: I was never involved with the development of the ATUSB hardware, firmware or driver software at any point in time. All credits go to Werner, Stefan and other contributors around ATUSB.

The IT security culture, hackers vs. industry consortia

Harald Welte — Tue, 06 Dec 2016 00:00:00 GMT

In a previous life I used to do a lot of IT security work, probably even at a time when most people had no idea what IT security actually is. I grew up with the Chaos Computer Club, as it was a great place to meet people with common interests, skills and ethics. People were hacking (aka 'doing security research') for fun, to grow their skills, to advance society, to point out corporate stupidities and to raise awareness about issues.

I've always shared any results worth noting with the general public. Whether it was in RFID security, on GSM security, TETRA security, etc.

Even more so, I always shared the tools, creating free software implementations of systems that - at that time - were very difficult to impossible to access unless you worked for the vendors of related device, who obviously had a different agenda then to disclose security concerns to the general public.

Publishing security related findings at related conferences can be interpreted in two ways:

On the one hand, presenting at a major event will add to your credibility and reputation. That's a nice byproduct, but that shouldn't be the primarily reason, unless you're some kind of a egocentric stage addict.

On the other hand, presenting findings or giving any kind of presentation or lecture at an event is a statement of support for that event. When I submit a presentation at a given event, I think carefully if that topic actually matches the event.

The reason that I didn't submit any talks in recent years at CCC events is not that I didn't do technically exciting stuff that I could talk about - or that I wouldn't have the reputation that would make people consider my submission in the programme committee. I just thought there was nothing in my work relevant enough to bother the CCC attendees with.

So when Holger 'zecke' Freyther and I chose to present about our recent journeys into exploring modern cellular modems at the annual Chaos Communications Congress, we did so because the CCC Congress is the right audience for this talk. We did so, because we think the people there are the kind of community of like-minded spirits that we would like to contribute to. Whom we would like to give something back, for the many years of excellent presentations and conversations had.

So far so good.

However, in 2016, something happened that I haven't seen yet in my 17 years of speaking at Free Software, Linux, IT Security and other conferences: A select industry group (in this case the GSMA) asking me out of the blue to give them the talk one month in advance at a private industry event.

I could hardly believe it. How could they? Who am I? Am I spending sleepless nights and non-existing spare time into security research of cellular modems to give a free presentation to corporate guys at a closed industry meeting? The same kind of industries that create the problems in the first place, and who don't get their act together in building secure devices that respect people's privacy? Certainly not. I spend sleepless nights of hacking because I want to share the results with my friends. To share it with people who have the same passion, whom I respect and trust. To help my fellow hackers to understand technology one step more.

If that kind of request to undermine the researcher/authors initial publication among friends is happening to me, I'm quite sure it must be happening to other speakers at the 33C3 or other events, too. And that makes me very sad. I think the initial publication is something that connects the speaker/author with his audience.

Let's hope the researchers/hackers/speakers have sufficiently strong ethics to refuse such requests. If certain findings are initially published at a certain conference, then that is the initial publication. Period. Sure, you can ask afterwards if an author wants to repeat the presentation (or a similar one) at other events. But pre-empting the initial publication? Certainly not with me.

I offered the GSMA that I could talk on the importance of having FOSS implementations of cellular protocol stacks as enabler for security research, but apparently this was not to their interest. Seems like all they wanted is an exclusive heads-up on work they neither commissioned or supported in any other way.

And btw, I don't think what Holger and I will present about is all that exciting in the first place. More or less the standard kind of security nightmares. By now we are all so numbed down by nobody considering security and/or privacy in design of IT systems, that is is hardly any news. IoT how it is done so far might very well be the doom of mankind. An unstoppable tsunami of insecure and privacy-invading devices, built on ever more complex technology with way too many security issues. We shall henceforth call IoT the Industry of Thoughtlessness.

Open Hardware miniPCIe WWAN modem USB breakout board released

Harald Welte — Thu, 24 Nov 2016 16:00:00 GMT

There are plenty of cellular modems on the market in the mPCIe form factor.

Playing with such modems is reasonably easy, you can simply insert them in a mPCIe slot of a laptop or an embedded device (soekris, pc-engines or the like).

However, many of those modems actually export interesting signals like digital PCM audio or UART ports on some of the mPCIe pins, both in standard and in non-standard ways. Those signals are inaccessible in those embedded devices or in your laptop.

So I built a small break-out board which performs the basic function of exposing the mPCIe USB signals on a USB mini-B socket, providing power supply to the mPCIe modem, offering a SIM card slot at the bottom, and exposing all additional pins of the mPCIe header on a standard 2.54mm pitch header for further experimentation.

The design of the board (including schematics and PCB layout design files) is available as open hardware under CC-BY-SA license terms. For more information see http://osmocom.org/projects/mpcie-breakout/wiki

If you don't want to build your own board, fully assembled and tested boards are available from http://shop.sysmocom.de/products/minipcie-wwan-modem-usb-break-out-board

Open Hardware Multi-Voltage USB UART board released

Harald Welte — Thu, 24 Nov 2016 16:00:00 GMT

During the past 16 years I have been playing a lot with a variety of embedded devices.

One of the most important tasks for debugging or analyzing embedded devices is usually to get access to the serial console on the UART of the device. That UART is often exposed at whatever logic level the main CPU/SOC/uC is running on. For 5V and 3.3V that is easy, but for ever more and more unusual voltages I always had to build a custom cable or a custom level shifter.

In 2016, I finally couldn't resist any longer and built a multi-voltage USB UART adapter.

This board exposes two UARTs at a user-selectable voltage of 1.8, 2.3, 2.5, 2.8, 3.0 or 3.3V. It can also use whatever other logic voltage between 1.8 and 3.3V, if it can source a reference of that voltage from the target embedded board.

Rather than just building one for myself, I released the design as open hardware under CC-BY-SA license terms. Full schematics + PCB layout design files are available. For more information see http://osmocom.org/projects/mv-uart/wiki

In case you don't want to build it from scratch, ready-made machine assembled boards are also made available from http://shop.sysmocom.de/products/multi-voltage-usb-dual-uart

Going to attend Electromagnetic Field 2016

Harald Welte — Sat, 23 Jul 2016 08:00:00 GMT

Based on some encouragement from friends as well as my desire to find more time again to hang out at community events, I decided to attend Electromagnetic Field 2016 held in Guildford, UK from August 5th through 7th.

As I typically don't like just attending an event without contributing to it in some form, I submitted a couple of talks / workshops, all of which were accepted:

An overview talk about the Osmocom project
A Workshop on running your own cellular network using OpenBSC and related Osmocom software
A Workshop on tracing (U)SIM card communication using Osmocom SIMtrace

I believe the detailed schedule is still in the works, as I haven't yet been able to find any on the event website.

Looking forward to having a great time at EMF 2016. After attending Dutch and German hacker camps for almost 20 years, let's see how the Brits go about it!

EC-GSM-IoT: Enhanced Coverage GSM for IoT

Harald Welte — Sat, 23 Jul 2016 04:00:00 GMT

In private conversation, Holger mentioned EC-GSM-IoT to me, and I had to dig a bit into it. It was introduced in Release 13, but if you do a web search for it, you find surprisingly little information beyond press releases with absolutely zero information content and no "further reading".

The primary reason for this seems to be that the feature was called EC-EGPRS until the very late stages, when it was renamed for - believe it or not - marketing reasons.

So when searching for the right term, you actually find specification references and change requests in the 3GPP document archives.

I tried to get a very brief overview, and from what I could find, it is centered around GERAN extension in the following ways:

EC-EGPRS goal: Improve coverage by 20dB
- New single-burst coding schemes
- Blind Physical Layer Repetitions where bursts are repeated up to 28 times without feedback from remote end
  - transmitter maintains phase coherency
  - receiver uses processing gain (like incremental redundancy?)
- New logical channel types (EC-BCCH, EC-PCH, EC-AGC, EC-RACH, ...)
- New RLC/MAC layer messages for the EC-PDCH communication
Power Efficient Operation (PEO)
- Introduction of eDRX (extended DRX) to allow for PCH listening intervals from minutes up to a hour
- Relaxed Idle Mode: Important to camp on a cell, not best cell. Reduces neighbor cell monitoring requirements

In terms of required modifications to an existing GSM/EDGE implementation, there will be (at least):

changes to the PHY layer regarding new coding schemes, logical channels and burst scheduling / re-transmissions
changes to the RLC/MAC layer in the PCU to implement the new EC specific message types and procedures
changes to the BTS and BSC in terms of paging in eDRX

In case you're interested in more pointers on technical details, check out the links provided at https://osmocom.org/issues/1780

It remains to be seen how widely this will be adopted. Rolling this cange out on moderm base station hardware seems technicalyl simple - but it remains to be seen how many equipment makers implement it, and at what cost to the operators. But I think the key issue is whether or not the baseband chipset makers (Intel, Qualcomm, Mediatek, ...) will implement it anytime soon on the device side.

There are no plans on implementing any of this in the Osmocom stack as of now,but in case anyone was interested in working on this, feel free to contact us on the osmocom-net-gprs@lists.osmocom.org mailing list.

Deeper ventures into Ericsson (Packet) Abis

Harald Welte — Sat, 16 Jul 2016 04:00:00 GMT

Some topics keep coming back, even a number of years after first having worked on them. And then you start to search online using your favorite search engine - and find your old posts on that subject are the most comprehensive publicly available information on the subject ;)

Back in 2011, I was working on some very basic support for Ericsson RBS2xxx GSM BTSs in OpenBSC. The major part of this was to find out the weird dynamic detection of the signalling timeslot, as well as the fully non-standard OM2000 protocol for OML. Once it reached the state of a 'proof-of-concept', work at this ceased and remained in a state where still lots of manual steps were involved in BTS bring-up.

I've recently picked this topic up again, resulting in some work-in-progress code in http://git.osmocom.org/openbsc/log/?h=laforge/om2000-fsm

Beyond classic E1 based A-bis support, I've also been looking (again) at Ericsson Packet Abis. Packet Abis is their understanding of Abis over IP. However, it is - again - much further from the 3GPP specifications than what we're used to in the Osmocom universe. Abis/IP as we know consists of:

RSL and OML over TCP (inside an IPA multiplex)
RTP streams for the user plane (voice)
Gb over IP (NS over UDP/IP), as te PCU is in the BTS.

In the Ericsson world, they decided to taka a much lower-layer approach and decided to

start with L2TP over IP (not the L2TP over UDP that many people know from VPNs)
use the IETF-standardized Pseudowire type for HDLC but use a frame format in violation of the IETF RFCs
Talk LAPD over L2TP for RSL and OML
Invent a new frame format for voice codec frames called TFP and feed that over L2TP
Invent a new frame format for the PCU-CCU communication called P-GSL and feed that over L2TP

I'm not yet sure if we want to fully support that protocol stack from OpenBSC and related projects, but in any case I've extende wireshark to decode such protocol traces properly by

Extending the L2TP dissector with Ericsson specific AVPs
Improving my earlier pakcet-ehdlc.c with better understanding of the protocol
Implementing a new TFP dissector from scratch
Implementing a new P-GSL dissector from scratch

The resulting work can be found at http://git.osmocom.org/wireshark/log/?h=laforge/ericsson-packet-abis in case anyone is interested. I've mostly been working with protocol traces from RBS2409 so far, and they are decoded quite nicely for RSL, OML, Voice and Packet data. As far as I know, the format of the STN / SIU of other BTS models is identical.

Is anyone out there in possession of Ericsson RBS2xxx RBSs interested in collboration on either a Packet Abis implementation, or an inteface of the E1 or packet based CCU-PCU interface to OsmoPCU?

Nuand abusing the term "Open Source" for non-free Software

Harald Welte — Wed, 01 Jun 2016 04:00:00 GMT

Back in late April, the well-known high-quality SDR hardware company Nuand published a blog post about an Open Source Release of a VHDL ADS-B receiver.

I was quite happy at that time about this, and bookmarked it for further investigation at some later point.

Today I actually looked at the source code, and more by coincidence noticed that the LICENSE file contains a license that is anything but Open Source: The license is a "free for evaluation only" license, and it is only valid if you run the code on an actual Nuand board.

Both of the above are clearly not compatible with any of the well-known and respected definitions of Open Source, particularly not the official Open Source Definition of the Open Source Initiative.

I cannot even start how much this makes me upset. This is once again openwashing, where something that clearly is not Free or Open Source Software is labelled and marketed as such.

I don't mind if an author chooses to license his work under a proprietary license. It is his choice to do so under the law, and it generally makes such software utterly unattractive to me. If others still want to use it, it is their decision. However, if somebody produces or releases non-free or proprietary software, then they should make that very clear and not mis-represent it as something that it clearly isn't!

Open-washing only confuses everyone, and it tries to market the respective company or product in a light that it doesn't deserve. I believe the proper English proverb is to adorn oneself with borrowed plumes.

I strongly believe the community must stand up against such practise and clearly voice that this is not something generally acceptable or tolerated within the Free and Open Source software world. It's sad that this is happening more frequently, like recently with OpenAirInterface (see related blog post).

I will definitely write an e-mail to Nuand management requesting to correct this mis-representation. If you agree with my posting, I'd appreciate if you would contact them, too.

Osmocom.org GTP-U kernel implementation merged mainline

Harald Welte — Thu, 26 May 2016 04:00:00 GMT

Have you ever used mobile data on your phone or using Tethering?

In packet-switched cellular networks (aka mobile data) from GPRS to EDGE, from UMTS to HSPA and all the way into modern LTE networks, there is a tunneling protocol called GTP (GPRS Tunneling Protocol).

This was the first cellular protocol that involved transport over TCP/IP, as opposed to all the ISDN/E1/T1/FrameRelay world with their weird protocol stacks. So it should have been something super easy to implement on and in Linux, and nobody should have had a reason to run a proprietary GGSN, ever.

However, the cellular telecom world lives in a different universe, and to this day you can be safe to assume that all production GGSNs are proprietary hardware and/or software :(

In 2002, Jens Jakobsen at Mondru AB released the initial version of OpenGGSN, a userspace implementation of this tunneling protocol and the GGSN network element. Development however ceased in 2005, and we at the Osmocom project thus adopted OpenGGSN maintenance in 2016.

Having a userspace implementation of any tunneling protocol of course only works for relatively low bandwidth, due to the scheduling and memory-copying overhead between kernel, userspace, and kernel again.

So OpenGGSN might have been useful for early GPRS networks where the maximum data rate per subscriber is in the hundreds of kilobits, but it certainly is not possible for any real operator, particularly not at today's data rates.

That's why for decades, all commonly used IP tunneling protocols have been implemented inside the Linux kernel, which has some tunneling infrastructure used with tunnels like IP-IP, SIT, GRE, PPTP, L2TP and others.

But then again, the cellular world lives in a universe where Free and Open Source Software didn't exit until OpenBTS and OpenBSC changed all o that from 2008 onwards. So nobody ever bothered to add GTP support to the in-kernel tunneling framework.

In 2012, I started an in-kernel implementation of GTP-U (the user plane with actual user IP data) as part of my work at sysmocom. My former netfilter colleague and current netfilter core team leader Pablo Neira was contracted to bring it further along, but unfortunately the customer project funding the effort was discontinued, and we didn't have time to complete it.

Luckily, in 2015 Andreas Schultz of Travelping came around and has forward-ported the old code to a more modern kernel, fixed the numerous bugs and started to test and use it. He also kept pushing Pablo and me for review and submission, thanks for that!

Finally, in May 2016, the code was merged into the mainline kernel, and now every upcoming version of the Linux kernel will have a fast and efficient in-kernel implementation of GTP-U. It is configured via netlink from userspace, where you are expected to run a corresponding daemon for the control plane, such as either OpenGGSN, or the new GGSN + PDN-GW implementation in Erlang called erGW.

You can find the kernel code at drivers/net/gtp.c, and the userspace netlink library code (libgtpnl) at git.osmocom.org.

I haven't done actual benchmarking of the performance that you can get on modern x86 hardware with this, but I would expect it to be the same of what you can also get from other similar in-kernel tunneling implementations.

Now that the cellular industry has failed for decades to realize how easy and little effort would have been needed to have a fast and inexpensive GGSN around, let's see if now that other people did it for them, there will be some adoption.

If you're interested in testing or running a GGSN or PDN-GW and become an early adopter, feel free to reach out to Andreas, Pablo and/or me. The osmocom-net-gprs mailing list might be a good way to discuss further development and/or testing.

Slovenian student sentenced for detecting TETRA flaws using OsmocomTETRA

Harald Welte — Sat, 21 May 2016 16:00:00 GMT

According to some news report, including this report at softpedia, a 26 year old student at the Faculty of Criminal Justice and Security in Maribor, Slovenia has received a suspended prison sentence for finding flaws in Slovenian police and army TETRA network using OsmocomTETRA

As the Osmocom project leader and main author of OsmocomTETRA, this is highly disturbing news to me. OsmocomTETRA was precisely developed to enable people to perform research and analysis in TETRA networks, and to audit their safe and secure configuration.

If a TETRA network (like any other network) is configured with broken security, then the people responsible for configuring and operating that network are to be blamed, and not the researcher who invests his personal time and effort into demonstrating that police radio communications safety is broken. On the outside, the court sentence really sounds like "shoot the messenger". They should instead have jailed the people responsible for deploying such an insecure network in the first place, as well as those responsible for not doing the most basic air-interface interception tests before putting such a network into production.

According to all reports, the student had shared the results of his research with the authorities and there are public detailed reports from 2015, like the report (in Slovenian) at https://podcrto.si/vdor-v-komunikacijo-policije-razkril-hude-varnostne-ranljivosti-sistema-tetra/.

The statement that he should have asked the authorities for permission before starting his research is moot. I've seen many such cases and you would normally never get permission to do this, or you would most likely get no response from the (in)competent authorities in the first place.

From my point of view, they should give the student a medal of honor, instead of sentencing him. He has provided a significant service to the security of the public sector communications in his country.

To be fair, the news report also indicates that there were other charges involved, like impersonating a police officer. I can of course not comment on those.

Please note that I do not know the student or his research first-hand, nor did I know any of his actions or was involved in them. OsmocomTETRA is a Free / Open Source Software project available to anyone in source code form. It is a vital tool in demonstrating the lack of security in many TETRA networks, whether networks for public safety or private networks.

Developers wanted for Osmocom GSM related work

Harald Welte — Sun, 01 May 2016 16:00:00 GMT

Right now I'm feeling sad. I really shouldn't, but I still do.

Many years ago I started OpenBSC and Osmocom in order to bring Free Software into an area where it barely existed before: Cellular Infrastructure. For the first few years, it was "just for fun", without any professional users. A FOSS project by enthusiasts. Then we got some commercial / professional users, and with them funding, paying for e.g. Holger and my freelance work. Still, implementing all protocol stacks, interfaces and functional elements of GSM and GPRS from the radio network to the core network is something that large corporations typically spend hundreds of man-years on. So funding for Osmocom GSM implementations was always short, and we always tried to make the best out of it.

After Holger and I started sysmocom in 2011, we had a chance to use funds from BTS sales to hire more developers, and we were growing our team of developers. We finally could pay some developers other than ourselves from working on Free Software cellular network infrastructure.

In 2014 and 2015, sysmocom got side-tracked with some projects where Osmocom and the cellular network was only one small part of a much larger scope. In Q4/2015 and in 2016, we are back on track with focussing 100% at Osmocom projects, which you can probably see by a lot more associated commits to the respective project repositories.

By now, we are in the lucky situation that the work we've done in the Osmocom project on providing Free Software implementations of cellular technologies like GSM, GPRS, EDGE and now also UMTS is receiving a lot of attention. This attention translates into companies approaching us (particularly at sysmocom) regarding funding for implementing new features, fixing existing bugs and short-comings, etc. As part of that, we can even work on much needed infrastructural changes in the software.

So now we are in the opposite situation: There's a lot of interest in funding Osmocom work, but there are few people in the Osmocom community interested and/or capable to follow-up to that. Some of the early contributors have moved into other areas, and are now working on proprietary cellular stacks at large multi-national corporations. Some others think of GSM as a fun hobby and want to keep it that way.

At sysmocom, we are trying hard to do what we can to keep up with the demand. We've been looking to add people to our staff, but right now we are struggling only to compensate for the regular fluctuation of employees (i.e. keep the team size as is), let alone actually adding new members to our team to help to move free software cellular networks ahead.

I am struggling to understand why that is. I think Free Software in cellular communications is one of the most interesting and challenging frontiers for Free Software to work on. And there are many FOSS developers who love nothing more than to conquer new areas of technology.

At sysmocom, we can now offer what would have been my personal dream job for many years:

paid work on Free Software that is available to the general public, rather than something only of value to the employer
interesting technical challenges in an area of technology where you will not find the answer to all your problems on stackoverflow or the like
work in a small company consisting almost entirely only of die-hard engineers, without corporate managers, marketing departments, etc.
work in an environment free of Microsoft and Apple software or cloud services; use exclusively Free Software to get your work done

I would hope that more developers would appreciate such an environment. If you're interested in helping FOSS cellular networks ahead, feel free to have a look at http://sysmocom.de/jobs or contact us at jobs@sysmocom.de. Together, we can try to move Free Software for mobile communications to the next level!

You can now install a GSM network using apt-get

Harald Welte — Sun, 27 Mar 2016 16:00:00 GMT

This is great news: You can now install a GSM network using apt-get!

Thanks to the efforts of Debian developer Ruben Undheim, there's now an OpenBSC (with all its flavors like OsmoBSC, OsmoNITB, OsmoSGSN, ...) package in the official Debian repository.

Here is the link to the e-mail indicating acceptance into Debian: https://tracker.debian.org/news/755641

I think for the past many years into the OpenBSC (and wider Osmocom) projects I always assumed that distribution packaging is not really something all that important, as all the people using OpenBSC surely would be technical enough to build it from the source. And in fact, I believe that building from source brings you one step closer to actually modifying the code, and thus contribution.

Nevertheless, the project has matured to a point where it is not used only by developers anymore, and particularly also (god beware) by people with limited experience with Linux in general. That such people still exist is surprisingly hard to realize for somebody like myself who has spent more than 20 years in Linux land by now.

So all in all, today I think that having packages in a Distribution like Debian actually is important for the further adoption of the project - pretty much like I believe that more and better public documentation is.

Looking forward to seeing the first bug reports reported through bugs.debian.org rather than https://projects.osmocom.org/ . Once that happens, we know that people are actually using the official Debian packages.

As an unrelated side note, the Osmocom project now also has nightly builds available for Debian 7.0, Debian 8.0 and Ubunut 14.04 on both i586 and x86_64 architecture from https://build.opensuse.org/project/show/network:osmocom:nightly. The nightly builds are for people who want to stay on the bleeding edge of the code, but who don't want to go through building everything from scratch. See Holgers post on the openbsc mailing list for more information.

Open Source mobile communications, security research and contributions

Harald Welte — Mon, 14 Mar 2016 16:00:00 GMT

While preparing my presentation for the Troopers 2016 TelcoSecDay I was thinking once again about the importance of having FOSS implementations of cellular protocol stacks, interfaces and network elements in order to enable security researches (aka Hackers) to work on improving security in mobile communications.

From the very beginning, this was the motivation of creating OpenBSC and OsmocomBB: To enable more research in this area, to make it at least in some ways easier to work in this field. To close a little bit of the massive gap on how easy it is to do applied security research (aka hacking) in the TCP/IP/Internet world vs. the cellular world.

We have definitely succeeded in that. Many people have successfully the various Osmocom projects in order to do cellular security research, and I'm very happy about that.

However, there is a back-side to that, which I'm less happy about. In those past eight years, we have not managed to attract significant amount of contributions to the Osmocom projects from those people that benefit most from it: Neither from those very security researchers that use it in the first place, nor from the Telecom industry as a whole.

I can understand that the large telecom equipment suppliers may think that FOSS implementations are somewhat a competition and thus might not be particularly enthusiastic about contributing. However, the story for the cellular operators and the IT security crowd is definitely quite different. They should have no good reason not to contribute.

So as a result of that, we still have a relatively small amount of people contributing to Osmocom projects, which is a pity. They can currently be divided into two groups:

the enthusiasts: People contributing because they are enthusiastic about cellular protocols and technologies.
the commercial users, who operate 2G/2.5G networks based on the Osmocom protocol stack and who either contribute directly or fund development work at sysmocom. They typically operate small/private networks, so if they want data, they simply use Wifi. There's thus not a big interest or need in 3G or 4G technologies.

On the other hand, the security folks would love to have 3G and 4G implementations that they could use to talk to either mobile devices over a radio interface, or towards the wired infrastructure components in the radio access and core networks. But we don't see significant contributions from that sphere, and I wonder why that is.

At least that part of the IT security industry that I know typically works with very comfortable budgets and profit rates, and investing in better infrastructure/tools is not charity anyway, but an actual investment into working more efficiently and/or extending the possible scope of related pen-testing or audits.

So it seems we might want to think what we could do in order to motivate such interested potential users of FOSS 3G/4G to contribute to it by either writing code or funding associated developments...

If you have any thoughts on that, feel free to share them with me by e-mail to laforge@gnumonks.org.

TelcoSecDay 2016: Open Source Network Elements for Security Analysis of Mobile Networks

Harald Welte — Mon, 14 Mar 2016 16:00:00 GMT

Today I had the pleasure of presenting about Open Source Network Elements for Security Analysis of Mobile Networks at the Troopers 2016 TelcoSecDay.

The main topics addressed by this presentation are:

Importance of Free and Open Source Software implementations of cellular network protocol stacks / interfaces / network elements for applied telecom security research
The progress we've made at Osmocom over the last eight years.
An overview about our current efforts to implement at 3G Network similar to the existing 2G/2.5G/2.75G implementations.

There are no audio or video recordings of this session.

Slides are available at https://gitea.osmocom.org/laforge/laforge-slides/src/branch/master/2016/telcosecday

Osmocom.org migrating to redmine

Harald Welte — Sat, 20 Feb 2016 16:00:00 GMT

In 2008, we started bs11-abis, which was shortly after renamed to OpenBSC. At the time it seemed like a good idea to use trac as the project management system, to have a wiki and an issue tracker.

When further Osmocom projects like OsmocomBB, OsmocomTETRA etc. came around, we simply replicated that infrastructure: Another trac instance with the same theme, and a shared password file.

The problem with this (and possibly the way we used it) is:

it doesn't scale, as creating projects is manual, requires a sysadmin and is time-consuming. This meant e.g. SIMtrace was just a wiki page in the OsmocomBB trac installation + associated http redirect, causing some confusion.
issues can not easily be moved from one project to another, or have cross-project relationships (like, depend on an issue in another project)
we had to use an external planet in order to aggregate the blog of each of the trac instances
user account management the way we did it required shell access to the machine, meaning user account applications got dropped due to the effort involved. My apologies for that.

Especially the lack of being able to move pages and tickets between trac's has resulted in a suboptimal use of the tools. If we first write code as part of OpenBSC and then move it to libosmocore, the associated issues + wiki pages should be moved to a new project.

At the same time, for the last 5 years we've been successfully using redmine inside sysmocom to keep track of many dozens of internal projects.

So now, finally, we (zecke, tnt, myself) have taken up the task to migrate the osmocom.org projects into redmine. You can see the current status at http://projects.osmocom.org/. We could create a more comprehensive project hierarchy, and give libosmocore, SIMtrace, OsmoSGSN and many others their own project.

Thanks to zecke for taking care of the installation/sysadmin part and the initial conversion!

Unfortunately the conversion from trac to redmine wiki syntax (and structure) was not as automatic and straight-forward as one would have hoped. But after spending one entire day going through the most important wiki pages, things are looking much better now. As a side effect, I have had a more comprehensive look into the history of all of our projects than ever before :)

Still, a lot of clean-up and improvement is needed until I'm happy, particularly splitting the OpenBSC wiki into separate OsmoBSC, OsmoNITB, OsmoBTS, OsmoPCU and OsmoSGSN wiki's is probably still going to take some time.

If you would like to help out, feel free to register an account on projects.osmocom.org (if you don't already have one from the old trac projects) and mail me for write access to the project(s) of your choice.

Possible tasks include

putting pages into a more hierarchic structure (there's a parent/child relationship in redmine wikis)
fixing broken links due to page renames / wiki page moves
creating a new redmine 'Project' for your favorite tool that has a git repo on http://git.osmocom.org/ and writing some (at least initial) documentation about it.

You don't need to be a software developer for that!

Some update on recent OsmoBTS changes

Harald Welte — Fri, 19 Feb 2016 16:00:00 GMT

After quite some time of gradual bug fixing and improvement, there have been quite some significant changes being made in OsmoBTS over the last months.

Just a quick reminder: In Fall 2015 we finally merged the long-pending L1SAP changes originally developed by Jolly, introducing a new intermediate common interface between the generic part of OsmoBTS, and the hardware/PHY specific part. This enabled a clean structure between osmo-bts-sysmo (what we use on the sysmoBTS) and osmo-bts-trx (what people with general-purpose SDR hardware use).

The L1SAP changes had some fall-out that needed to be fixed, not a big surprise with any change that big.

More recently however, three larger changes were introduced:

phy_link / phy_instance abstraction

There now is the concept of a phy_link, each of which can have multiple phy_instances. Each instance represents one baseband transceiver, i.e. a software or hardware unit driving a TRX inside a BTS.

Every BTS model has been converted to use this new abstraction layer.

proper Multi-TRX support

Based on the above phy_link/phy_instance infrastructure, one can map each phy_instance to one TRX by means of the VTY / configuration file.

The core of OsmoBTS now supports any number of TRXs, leading to flexible Multi-TRX support.

OCTPHY support

A Canadian company called Octasic has been developing a custom GSM PHY for their custom multi-core DSP architecture (OCTDSP). Rather than re-inventing the wheel for everything on top of the PHY, they chose to integrate OsmoBTS on top of it. I've been working at sysmocom on integrating their initial code into OsmoBTS, rendering a new osmo-bts-octphy backend.

This back-end has also recently been ported to the phy_link/phy_instance API and is Multi-TRX ready. You can both run multiple TRX in one DSP, as well as have multiple DSPs in one BTS, paving the road for scalability.

osmo-bts-octphy is now part of OsmoBTS master.

Corresponding changes to OsmoPCU (for full GPRS support on OCTPHY) are currently been worked on by Max at sysmocom.

Litecell 1.5 PHY support

Another Canadian company (Nutaq/Nuran) has been building a new BTS called Litecell 1.5. They also implemented OsmoBTS support, based on the osmo-bts-sysmo code. We've been able to integrate that code with the above-mentioned phy_link/phy_interface in order to support the MultiTRX capability of this hardware.

Litecell 1.5 MultiTRX capability has also been integrated with OsmoPCU.

osmo-bts-litecell15 is now part of OsmoBTS master.

Summary

2016 starts as the OsmoBTS year of MultiTRX.
2016 also starts as a year of many more hardware choices for OsmoBTS
we see more commercial adoption of OsmoBTS outside of the traditional options of sysmocom and Fairwaves

netdevconf 1.1: Osmocom kernel-level GTP implementation

Harald Welte — Wed, 10 Feb 2016 16:00:00 GMT

Today I had the pleasure of co-presenting with Andreas Schultz at netdevconf 1.1 about the Osmocom kernel-level GTP implementation.

The video recording is available from https://www.youtube.com/watch?v=puCMipd8fck

Slides are available at https://gitea.osmocom.org/laforge/laforge-slides/src/branch/master/2016/netdevconf-gtp

netdevconf 1.1: Running cellular infrastructure on Linux

Harald Welte — Tue, 09 Feb 2016 16:00:00 GMT

Today I had the pleasure of presenting at netdevconf 1.1 a tutorial about Running cellular infrastructure on Linux. The tutorial is intended to guide you through the process of setting up + configuring yur own minimal private GSM+GPRS network.

The video recording is available from https://www.youtube.com/watch?v=I4i2Gy4JhDo

Slides are available at https://gitea.osmocom.org/laforge/laforge-slides/src/branch/master/2016/netdevconf-osmocom

32C3 is over, GSM and GPRS was running fine, osmo-iuh progress

Harald Welte — Wed, 30 Dec 2015 16:00:00 GMT

The 32C3 GSM Network

32C3 was great from the Osmocom perspective: We could again run our own cellular network at the event in order to perform load testing with real users. We had 7 BTSs running, each with a single TRX. What was new compared to previous years:

OsmoPCU is significantly more robust and stable due to the efforts of Jacob Erlbeck at sysmocom. This means that GPRS is now actually still usable in severe overload situations, like 1000 subscribers sharing only very few kilobits. Of course it will be slow, but at least data still passes through as much as that's possible.
We were using half-rate traffic channels from day 2 onwards, in order to enhance capacity. Phones supporting AMR-HR would use that, but then there are lots of old phones that only do classic HR (v1). OsmoNITB with internal MNCC handler supports TCH/H with HR and AMR for at least five years, but the particular combination of OsmoBTS + OsmoNITB + lcr (all master branches) was not yet deployed at previous CCC event networks so far.

Being forced to provide classic HR codec actually revealed several bugs in the existing code:

OsmoBTS (at least with the sysmoBTS hardware) is using bit ordering that is not compliant to what the spec says on how GSM-HR frames should be put into RTP frames. We didn't realize this so far, as handing frames from one sysmoBTS to another sysmoBTS of course works, as both use the same (wrong) bit ordering.
The ETSI reference implementation of the HR codec has lots of global/static variables, and thus doesn't really support running multiple transcoders in parallel. This is however what lcr was trying (and needing) to do, and it of course failed as state from one transcoder instance was leaking into another. The problem is simple, but the solution not so simple. If you want to avoid re-structuring the entire code in very intrusive ways or running one thread per transcoder instance, then the only solution was to basically memcpy() the entire data section of the transcoding library every time you switch the state from one transcoder instance to the other. It's surprisingly difficult to learn the start + size of that data section at runtime in a portable way, though.

Thanks to our resident voice codec expert Sylvain for debugging and fixing the above two problems.

Thanks also to Daniel and Ulli for taking care of the actual logistics of bringing + installing (+ later unmounting) all associated equipment.

Thanks furthermore to Kevin who has been patiently handling the 'Level 2 Support' cases of people with various problems ending up in the GSM room.

It's great that there is a team taking care of those real-world test networks. We learn a lot more about our software under heavy load situations this way.

osmo-iuh progress + talk

I've been focussing basically full day (and night) over the week ahead of Christmas and during Christmas to bring the osmo-iuh code into a state where we could do a end-to-end demo with a regular phone + hNodeB + osmo-hnbgw + osmo-sgsn + openggsn. Unfortunately I only got it up to the point where we do the PDP CONTEXT ACTIVATION on the signalling plane, with no actual user data going back and forth. And then, for strange reasons, I couldn't even demo that at the end of the talk. Well, in either case, the code has made much progress.

The video of the talk can be found at https://media.ccc.de/v/32c3-7412-running_your_own_3g_3_5g_network#video

meeting friends

The annual CCC congress is always an event where you meet old friends and colleagues. It was great talking to Stefan, Dimitri, Kevin, Nico, Sylvain, Jochen, Sec, Schneider, bunnie and many other hackers. After the event is over, I wish I could continue working together with all those folks the rest of the year, too :/

Some people have been missed dearly. Absence from the CCC congress is not acceptable. You know who you are, if you're reading this ;)

32C3: Running your own 3G/3.5G cellular network

Harald Welte — Sat, 26 Dec 2015 16:00:00 GMT

Today I had the pleasure of presenting at 32C3 about Running your own 3G/3.5G cellular network. The tutorial covers the ongoing effort of creating a HNB-GW and Iuh/IuCS/IuPS support as part of the Osmocom project.

The video recording is available from https://media.ccc.de/v/32c3-7412-running_your_own_3g_3_5g_network

Slides are available at https://gitea.osmocom.org/laforge/laforge-slides/raw/branch/master/2015/osmo_iuh/osmo_iuh.pdf

Anyone interested in supporting SMPP interworking at 32C3?

Harald Welte — Sat, 05 Dec 2015 16:00:00 GMT

Sylvain brought this up yesterday: Wouldn't it be nice to have some degree of SMS interfacing from OpenBSC/OsmoNITB to the real world at 32C3? It is something that we've never tried so far, and thus definitely worthy of testing.

Of course, full interworking is not possible without assigning public MSISDN to all internal subscribers / 'extensions' how we call them.

But what would most certainly work is to have at least outbound SMS working by means of an external SMPP interface.

The OsmoNITB-internal SMSC speaks SMPP already (in the SMSC role), so we would need to implement some small amount of glue logic that behaves as ESME (external SMS entity) towards both OsmoNITB as well as some public SMS operator/reseller that speaks SMPP again.

Now of course, sending SMS to public operators doesn't come for free. So in case anyone reading this has access to SMPP at public operators, resellers, SMS hubs, it would be interesting to see if there is a chance for some funding/sponsoring of that experiment.

Feel free to contact me if you see a way to make this happen.

python-libsmpp works great with OsmoNITB

Harald Welte — Fri, 04 Dec 2015 16:00:00 GMT

Since 2012 we have support for SMPP in OsmoNITB (the network-in-the-box version of OpenBSC). So far I've only used it from C and Erlang code.

Yesterday I gave python-smpplib from https://github.com/podshumok/python-smpplib a try and it worked like a charm. Of course one has to get the details right (like numbering plan indication).

In case anyone is interested in interfacing OsmoNITB SMPP from python, I've put a working example to send SMS at http://cgit.osmocom.org/mncc-python/tree/smpp_test.py

Python tool to talk to OsmoNITB MNCC interface

Harald Welte — Tue, 01 Dec 2015 16:00:00 GMT

I've been working on a small python tool that can be used to attach to the MNCC interface of OsmoNITB. It implements the 04.08 CC state machine with our MNCC primitives, including support for RTP bridge mode of the voice streams.

The immediate first use case for this was to be able to generate MT calls to a set of known MSISDNs and load all 14 TCH/H channels of a single-TRX BTS. It will connect the MT calls in pairs, so you end up with 7 MS-to-MS calls.

The first working version of the tool is available from

http://git.osmocom.org/mncc-python/
git clone git://git.osmocom.org/mncc-python

The code is pretty hacky in some places. That's partially due to the fact that I'm much more familiar in the C, Perl and Erlang world than in python. Still I thought it's a good idea to do it in python to enable more people to use/edit/contribute to it.

I'm happy for review / cleanup suggestion by people with more Python-foo than I have.

Architecturally, I decided to do things a bit erlang-like, where we have finite state machines in an actor models, and message passing between the actors. This is what happens with the GsmCallFsm()'s, which are created by the GsmCallConnector() representing both legs of a call and the MnccActor() that wraps the MNCC socket towards OsmoNITB.

The actual encoding/decoding of MNCC messages is auto-generated from the mncc header file #defines, enums and c-structures by means of ctypes code generation.

mncc_test.py currently drops you into a python shell where you can e.g. start more / new calls by calling functions like connect_call("7839", "3802") from that shell. Exiting the shell by quit() or Ctrl+C will terminate all call FSMs and terminate.

GSM test network at 32C3, after all

Harald Welte — Sun, 15 Nov 2015 16:00:00 GMT

Contrary to my blog post yesterday, it looks like we will have a private GSM network at the CCC congress again, after all.

It appears that Vodafone Germany (who was awarded the former DECT guard band in the 2015 spectrum auctions) is not yet using it in December, and they agreed that we can use it at the 32C3.

With this approval from Vodafone Germany we can now go to the regulator (BNetzA) and obtain the usual test license. Given that we used to get the license in the past, and that Vodafone has agreed, this should be a mere formality.

For the German language readers who appreciate the language of the administration, it will be a Frequenzzuteilung für Versuchszwecke im nichtöffentlichen mobilen Landfunk.

So thanks to Vodafone Germany, who enabled us at least this time to run a network again. By end of 2016 you can be sure they will have put their new spectrum to use, so I'm not that optimistic that this would be possible again.

No GSM test network at 32C3

Harald Welte — Sat, 14 Nov 2015 16:00:00 GMT

I currently don't assume that there will be a GSM network at the 32C3.

Ever since OpenBSC was created in 2008, the annual CCC congress was a great opportunity to test OpenBSC and related software with thousands of willing participants. In order to do so, we obtained a test licence from the German regulatory authority. This was never any problem, as there was a chunk of spectrum in the 1800 MHz GSM band that was not allocated to any commercial operator, the so-called DECT guard band. It's called that way as it was kept free in order to ensure there is no interference between 1800 MHz GSM and the neighboring DECT cordless telephones.

Over the decades, it was determined on a EU level that this guard band might not be necessary, or at least not if certain considerations are taken for BTSs deployed in that band.

When the German regulatory authority re-auctioned the GSM spectrum earlier this year, they decided to also auction the frequencies of the former DECT guard band. The DECT guard band was awarded to Vodafone.

This is a pity, as this means that people involved with cellular research or development of cellular technology now have it significantly harder to actually test their systems.

In some other EU member states it is easier, like in the Netherlands or the UK, where the DECT guard band was not treated like any other chunk of the GSM bands, but put under special rules. Not so in Germany.

To make a long story short: Without the explicit permission of any of the commercial mobile operators, it is not possible to run a test/experimental network like we used to ran at the annual CCC congress.

Given that

the event is held in the city center (where frequencies are typically used and re-used quite densely), and
an operator has nothing to gain from permitting us to test our open source GSM/GPRS implementations,

I think there is little chance that this will become a reality.

If anyone has really good contacts to the radio network planning team of a German mobile operator and wants to prove me wrong: Feel free to contact me by e-mail.

Thanks to everyone involved with the GSM team at the CCC events, particularly Holger Freyther, Daniel Willmann, Stefan Schmidt, Jan Luebbe, Peter Stuge, Sylvain Munaut, Kevin Redon, Andreas Eversberg, Ulli (and everyone else whom I may have forgot, my apologies). It's been a pleasure!

Thanks also to our friends at the POC (Phone Operation Center) who have provided interfacing to the DECT, ISDN, analog and VoIP network at the events. Thanks to roh for helping with our special patch requests. Thanks also to those entities and people who borrowed equipment (like BTSs) in the pre-sysmocom years.

So long, and thanks for all the fish!

Osmocom Berlin meetings

Harald Welte — Sat, 07 Nov 2015 16:00:00 GMT

Back in 2012, I started the idea of having a regular, bi-weekly meeting of people interested in mobile communications technology, not only strictly related to the Osmocom projects and software. This was initially called the Osmocom User Group Berlin. The meetings were held twice per month in the rooms of the Chaos Computer Club Berlin.

There are plenty of people that were or still are involved with Osmocom one way or another in Berlin. Think of zecke, alphaone, 2b-as, kevin, nion, max, prom, dexter, myself - just to name a few.

Over the years, I got "too busy" and was no longer able to attend regularly. Some people kept it alive (thanks to dexter!), but eventually they were discontinued in 2013.

Starting in October 2015, I started a revival of the meetings, two have been held already, the third is coming up next week on November 11.

I'm happy that I had the idea of re-starting the meeting. It's good to meet old friends and new people alike. Both times there actually were some new faces around, most of which even had a classic professional telecom background.

In order to emphasize the focus is strictly not on Osmocom alone ( particularly not about its users only), I decided to rename the event to the Osmocom Meeting Berlin.

If you're in Berlin and are interested in mobile communications technology on the protocol and radio side of things, feel free to join us next Wednesday.

Progress on the Linux kernel GTP code

Harald Welte — Sat, 07 Nov 2015 16:00:00 GMT

It is always sad if you start to develop some project and then never get around finishing it, as there are too many things to take care in parallel. But then, days only have 24 hours...

Back in 2012 I started to write some generic Linux kernel GTP tunneling code. GTP is the GPRS Tunneling Protocol, a protocol between core network elements in GPRS networks, later extended to be used in UMTS and even LTE networks.

GTP is split in a control plane for management and the user plane carrying the actual user IP traffic of a mobile subscriber. So if you're reading this blog via a cellular interent connection, your data is carried in GTP-U within the cellular core network.

To me as a former Linux kernel networking developer, the user plane of GTP (GTP-U) had always belonged into kernel space. It is a tunneling protocol not too different from many other tunneling protocols that already exist (GRE, IPIP, L2TP, PPP, ...) and for the user plane, all it does is basically add a header in one direction and remove the header in the other direction. User data, particularly in networks with many subscribers and/or high bandwidth use.

Also, unlike many other telecom / cellular protocols, GTP is an IP-only protocol with no E1, Frame Relay or ATM legacy. It also has nothing to do with SS7, nor does it use ASN.1 syntax and/or some exotic encoding rules. In summary, it is nothing like any other GSM/3GPP protocol, and looks much more of what you're used from the IETF/Internet world.

Unfortunately I didn't get very far with my code back in 2012, but luckily Pablo Neira (one of my colleagues from netfilter/iptables days) picked it up and brought it along. However, for some time it has been stalled until recently it was thankfully picked up by Andreas Schultz and now receives some attention and discussion, with the clear intention to finish + submit it for mainline inclusion.

The code is now kept in a git repository at http://git.osmocom.org/osmo-gtp-kernel/

Thanks to Pablo and Andreas for picking this up, let's hope this is the last coding sprint before it goes mainline and gets actually used in production.

hardwear.io 2015 keynote: Running cellular infrastructure on Linux

Harald Welte — Thu, 01 Oct 2015 16:00:00 GMT

Today I had the pleasure of presenting at hardwear.io 2015 a keynote about Telecom Security - leassons learned... or not?.

Slides are available at https://gitea.osmocom.org/laforge/laforge-slides/raw/branch/master/2015/hardwear_io/keynote.pdf

OpenFest 2014: SIM card protocol tracing using Osmocom SIMtrace

Harald Welte — Sat, 01 Nov 2014 16:00:00 GMT

Today I had the pleasure of presenting at OpenFest 2014 a talk about Running cellular infrastructure on Linux.

The video recording is available from https://www.youtube.com/watch?v=n2GSn9HrEQs

Slides are available at https://gitea.osmocom.org/laforge/laforge-slides/raw/branch/master/2014/simtrace-openfest2014/simtrace.pdf

OpenFest 2014: Software Defined Radio using rtl-sdr

Harald Welte — Sat, 01 Nov 2014 16:00:00 GMT

Today I had the pleasure of presenting at OpenFest 2014 a talk about Software Defined Radio using rtl-sdr.

The video recording is available from https://www.youtube.com/watch?v=3LmAm9kRtgw

Slides are available at https://gitea.osmocom.org/laforge/laforge-slides/raw/branch/master/2014/rtlsdr-openfest2014/rtl-sdr.pdf

DORS/CLUC 2014: Keynote about Open Source Mobile Communications

Harald Welte — Sun, 15 Jun 2014 16:00:00 GMT

Today I had the pleasure of presenting at DORS/CLUC 2014 a keynote about Osmocom.org - Open Source Mobile Communications.

Slides are available at https://gitea.osmocom.org/laforge/laforge-slides/raw/branch/master/2014/osmocom-dorscluc2014/osmocom-overview.pdf

DORS/CLUC 2014: Keynote about OpenBSC - running your own GSM network

Harald Welte — Sun, 15 Jun 2014 16:00:00 GMT

Today I had the pleasure of presenting at DORS/CLUC 2014 a talk about OpenBSC - running your own GSM network.

Slides are available at https://gitea.osmocom.org/laforge/laforge-slides/raw/branch/master/2014/openbsc-dorscluc2014/gsm.pdf

OsmoDevCon 2013 preparation update

Harald Welte — Thu, 28 Mar 2013 19:00:00 GMT

OsmoDevCon 2013 is getting closer every day, and I'm very much looking forward to meet the fellow developers of the various Osmcoom sub-projects. Organization-wise, the catering has now been sorted out, and Holger has managed to get a test license for two ARFCN from the regulatory body without any trouble.

This means that we're more or less all set. The key needs to be picked up from IN-Berlin, and we need to bring some extra extension cords, ethernet switch, power cords and other gear, but that's really only very minor tasks.

There's not as much formal schedule as we used to have last year, which is good as I hope it means we can focus on getting actual work done, as opposed to spending most of the time updating one another about our respective work and progress.

Short report on the first Osmocom User Group meeting in Bavaria

Harald Welte — Fri, 07 Sep 2012 19:00:00 GMT

It's already one week in the past, but I'm only now finding some time to report on the first Osmocom User Group meeting in Bavaria.

All-in-all, there were 6 people attending, some people already known in the community, but also two completely new faces, which is great.

Dieter gave us a tour of his large BTS equipment, including a Nokia Ultrasite and an Ericsson RBS 2206. We had an introduction round where the participants could get to know each other a bit. Finally, we spoke about a variety of topics, from OsmocomBB to SIMtrace, SIM/SAT/STK security, the CC32RS512 and of course OpenBSC and the sysmoBTS.

On the day after the meeting I also had the pleasure of attempting to get the RBS2206 working with OpenBSC. Unfortunately there was no success, but still a number of bugs in the OM2000 / RBS2000 code in OpenBSC that had been found and fixed.

I'd like to thank Dieter Spaar for organizing and hosting the event, taking care of the Bavarian sausage + cheese platter for lunch.

I did not create rtl-sdr / librtlsdr

Harald Welte — Thu, 06 Sep 2012 19:00:00 GMT

In recent weeks, the number of private e-mails I receive about rtl-sdr has increased significantly. This is odd for at least two reasons:

First, I didn't create rtl-sdr and was not involved in its creation with the tiny exception of writing an e4k tuner driver for osmo-sdr, which was then used in a variety of rtl-sdr software.

Second, you should never contact the (presumed) software author in a private e-mail, but use the respective project mailing list. There is a community of developers, contributors and users out there, and it is a waste of everyone's time if you communicate by 1:1 private e-mail rather than enlightening the mailing list.

We're now working on a UMA/GAN controller

Harald Welte — Sat, 23 Jun 2012 19:00:00 GMT

We've pondered it a couple of times in the past whether we should implement an UMA/GAN controller (UNC/GANC). GAN (formerly called UMA) is a method by which you can tunnel GSM/3GPP Layer3 signalling (Mobility Management, SMS, Call Control) over an IP based bearer such as 802.11 (WiFi).

The idea was that mobile phones that support both a GSM/3G radio as well as WiFi could then simply use WiFi to connect to their mobile operator. This has been deployed around 2007/2008 by some operators such as T-Mobile USA as well as Orange UK. Today it seems that not many operators have caught up and UMA/GAN is mostly a legacy technology, last but not least due to very few phones actually implementing it.

Nonetheless, there are some markets and applications where UMA/GAN is useful. We (Dieter and I) now have managed to secure a contract for an Osmocom implementation based on OpenBSC (and libosmogsm, libosmo-sccp, ...). The beauty is that from L3 up, it is just regular GSM, no change needed at all. Only the transport layer is different: IPsec with TCP + GAN is the bearer, instead of LAPDm/RSL in classic GSM networks.

Another good part unrelated to UMA/GAN is: This will finally force us to clean up the separation between the MSC and BSC part in OsmoNITB (in order to replace the BSC part with the GANC).

Progress has been good so far, the SEGW (IPsec with EAP-SIM) has been configured, and a simplistic start of a GAN protocol implementation gets us through DISCOVERY, REGISTRATION and up to the point where the MS is sending the LOCATION UPDATE message. If you are curious how the protocol actually looks like, I've attached a sample pcap file to the WRTU54G-TM page in the OpenBSC wiki. The source code can be found in the laforge/ganc branch of openbsc.git.

osmo-lea6t-gps timing module DIY kits available

Harald Welte — Sat, 19 May 2012 19:00:00 GMT

Due to lots of other work, it took quite some time between my initial blog post about the omso-lea6t-gps board and the point where we are able to offically sell kits in the sysmocom webshop. The primary reason is: The people for whom we primarily built the board (i.e. the Osmocom developers) all have one and are happy with it ;)

But repeated inquiries by e-mail and otherwise have shown there is more interest. However, for a hand ful of boards we cannot make an automated production run in a SMT assembly line. So for the time being, we are only selling DYI kits, consisting of a digikey-packaged component kit including all components, plus the PCB, as well as the LEA-6T module.

Anyone who is interested in such a timing module DIY kit can now order from the sysmocom webshop.

More information on the project, including design materials like schematics can be found at the Osmocom wiki.

OsmoSDR boards available for interested developers

Harald Welte — Thu, 17 May 2012 19:00:00 GMT

I've posted about this on the OsmoSDR blog, so there's no point in copy+pasting it here.

There are still boards available, so feel free to order if you are interested in yet another exciting Osmocom embedded hardware/firmware/driver/software project!

Some follow-up on the Osmocom Berlin meetings

Harald Welte — Sun, 06 May 2012 19:00:00 GMT

We've now had the first two incarnations of the Osmocom Berlin User Group Meeting. The start was great, and we had probably something around 10 attendees. Some were the usual suspects like the various Osmocom developers living in Berlin. But we also had a number of new people attending each of both of the meetings, which is good.

To my big surprise people are even flying in from other parts of Europe in order to be able to attend. Last time from Sweden, and for the next meeting some folks from the Netherlands have announced themselves.

To an even bigger surprise, the attendee from Sweden announced that he is working for an Ericsson research lab, and apparently they are using OsmocomBB quite a bit inside that lab. They think it's a great tool, and apparently nothing else with the same flexibility (i.e. full source code) is at their hands that can compete.

On the one hand it is surprising to see such a large traditional Telco supplier to start to use such amateur tools like OsmocomBB, which definitely have not had even a fraction of the testing (particularly with various operators in various countries) like the commercial protocol stacks.

On the other hand, if you think more about it, Ericsson is entirely a network equipment supplier today. They have spun off their baseband processor business to become part of ST-Ericsson, they have pulled out of Sony-Ericsson, sold their TEMS product line to Ascom and other bits and pieces to Tieto. So right now, if they need a MS-side protocol stack or engineering phones, they probably have to obtain what is available on the market. And that's unfortunately not all that great, as the products are either

Measurement devices aimed at mostly L1 testing / QA (Racal, Agilent, Rohde-Schwarz)
Trace mobiles primarily aimed at field testing (TEMS, Sagem OT) and while they provide traces they don't permit you to send arbitrary data or behave spec-incompliant
Mobile Phone development platforms (Qualcomm, MTK, Infinenon, ...) which don't necessarily give you the full source code to the stack, and are only available if you actually intend to build a handset

So all in all, the more I think about it, it is actually not too surprising that they ended up with OsmocomBB. It's free (as in free beer) and they get the full source code with it. You need a lot of skills and time to get it running and find your way around how to use it, but I guess if you're working in cellular protocols and embedded systems, it's not that hard.

Prototype smart card chips in DIL-40 case have arrived

Harald Welte — Sun, 08 Apr 2012 19:00:00 GMT

Finally, the first samples of the smart card chip (for the Osmocom CardOS project) have arrived. As opposed to the final smart cards, this one has been packaged in a DIL case instead of the usual thin credit-card sized plastic. The reason for this is quite simple: This way lots of I/O pins for debugging as well as JTAG can be accessible during COS development.

Here you can see the first incarnation of a veroboard connected to an adapter pcb inside an Omnikey smart card reader:

After confirming it worked, I soldered the wires directly to the adapter PCB, as can be seen here:

There is already a real PCB design that is currently manufactured, i.e. in a week or so there will be a picture of a clean, professionally-produced/etched PCB with all of the prototype pins exported.

In terms of the COS, I haven't done much more work than compared to the last posting, mainly due to a large number of other projects. But we will get there...

h-online article covering OpenBTS and OpenBSC

Harald Welte — Sun, 25 Mar 2012 19:00:00 GMT

You can find a 3-page article about OpenBTS, OpenBSC and related projects available from the h-online web site.

OsmoDevCon 2012 is over...

Harald Welte — Sun, 25 Mar 2012 19:00:00 GMT

We just finished the 4th and final day of the OsmoDevCon 2012. It contained four days of in-depth presentations and discussions related to Free Software communications systems, most notably OsmocomBB, OpenBSC, OpenBTS, OsmoNITB, SIMtrace, OsmoGMR, OsmoSDR, rtl-sdr and many more.

I think it was a great chance to make sure the key developers involved with those projects are up-to-date with what everyone else is hacking on. I was especially happy with the presentations of Holger's smalltalk implementation of certain GSM protocols/interfaces, and it seems my small informal Erlang intro has raised some interest.

If anything, the 4-day conference has shown that there is a massive amount of work going on in the various different projects, and that it has clearly grown beyond anything that a single person could still be involved in all the sub-projects.

Personally, I'm happy to see what has grown out of this "we have a BS-11, let's see what we can do with it" that Dieter and I started in 2008. Now we're no longer talking about BTS/A-bis/BSC, but about SS7, MSC, TCAP/MAP, SCCP, HLR, Erlang, smalltalk, DECT, SIM/USIM, COS, SDR, GMR/Thuraya, TETRA and more recently also femtocells as well as NodeBs.

In the spirit of that 2008 presentation Running your own GSM network using the BS-11, Dieter Spaar has now demonstrated his talk on Running your own UMTS network, using NSN or Ericsson NodeBs. I'm really excited to see where that will take us - despite the fact that due to the 5 MHz wide channels, it's pretty close to impossible to get the experimental spectrum licenses that most of us have been able to get in recent years for our work.

As an outlook, over the remaining year 2012, I see progress in the following areas:

osmo-nitb will get a VLR/HLR split (async database access)
we will build a stand-alone osmo-msc with A interface
the signerl TCAP/MAP implementations will be used in production
OsmoSDR firmware will be completed, the hardware will start shipping
a new card operating system (OsmoCOS) will emerge
a UMA gateway will be implemented
a Free Software GPRS/EDGE PCU and RLC/MAC implementation will appear
last but not least, sysmoBTS will start commercial shipment really soon now

I'd like to thank our host c-base for having us block their conference room for 4 days, as well as all attendees who have travelled from all parts of Europe, but even the United States and Russia to participate. There definitely will be another OsmoDevCon, though we don't know yet at which point in time.

Using a cheap (USD 20) DVB-T USB stick as SDR receiver for (not only) gnuradio

Harald Welte — Sat, 17 Mar 2012 19:00:00 GMT

Fellow Osmocom hacker Steve Markgraf has been working on what now seems to be the cheapest way to receive real-world radio signals for PC-based SDR like gnuradio: rtl-sdr. RTL refers to the RTL2832U chipset frequently used in such devices. It can be used to obtain 2.8 Ms/s of 8-bit I+Q samples.

Below is a picture (courtesy of Steve) how the hardware looks like:

Osmocom GPS timing source with u-blox LEA6-T

Harald Welte — Thu, 15 Mar 2012 19:00:00 GMT

Recently we have been looking for an inexpensive way to generate a high-accuracy clock source for E1 lines, as it is required by a number of classic BTSs that don't have a sufficiently accurate OCXO built-in.

Luckily, the Digium E1 cards like TE-410P have a timing connector, to which an 8.192 MHz signal can be injected. Unfortunately, there don't seem to be any OCXOs around for that frequency. That's where the u-blox LEA-6T comes into play: It has a configurable TIMEPULSE2 output that can generate any frequency up to 10 MHz. We use this in our board to generate 8.192 Mhz and want to feed that into the Digium card.

So all we had to do is build a small board that contains the module and connector for antenna input, clock output and the obligatory 2.5mm stereo jack for the OsmocomBB-style UART:

Thanks to Sylvain for doing the schematics/PCB design, and thanks to Pablo for writing the code to configurea and activate the 8.192MHz output.

Once the design is verified, the schematics + gerber will be available, as well as board from the sysmocom webshop.

The next project on the horizon: A Free Software CardOS

Harald Welte — Thu, 01 Mar 2012 19:00:00 GMT

Now that we have a 100% free software GSM protocol stack and baseband firmware for the network and mobile phone side, the only remaining proprietary part is the SIM card. And what is a SIM card? It's a small embedded computer / SoC with integrated flash + RAM.

Once again, like in many other areas of the telecommunications industry, development of Free Software has been hampered by lack of available register-level hardware documentation. Without such information, how should you be able to program? Hardware without such documentation is an insult to every software developer.

The next problem is that typically, the Card Operating System (COS) is written into mask ROM of the smartcard SoC. Making such a mask is quite expensive, and it means that for every software version, different silicon will have to be produced. So unless you are going to have millions of units in quantity, it is unlikely that it would make economic sense.

However, in recent years, purely flash based smartcard chips have been available and getting less and less expensive. However, none of them (like the Atmel AT90SC7272 or similar devices) have freely available documentation. Furthermore, availability on the open market is somewhat of a problem, mainly because they have been used extensively by people cracking encrypted satellite TV channels. In recent years, the smartcard industry is trying hard to cut any kind of supply to that group of users.

However, luckily, we now see small/independent chip design houses in China picking up and producing their own smartcard chips. They are not only cheaper, but they simply hand out the documentation to anyone who asks them. No questions asked, no NDA required. Welcome to the promised land! That's what Free Software developers like:

Free access to documentation without any confidentiality agreements
development samples available at the same price as quantity pricing later on
inexpensive development hardware with JTAG access
reference source code provided without NDA
they are happy that somebody wants to develop for their hardware

As you can see, I am quite enthusiastic about this. I like this no-bullshit approach. No stupid marketing and sales droids who charge ridiculous fees for proprietary development tools that are inflexible and force developers to use one particular OS/IDE/toolchain.

I'm not sure how much time there will be, given the multitude of other projects that are all asking for attention. However, I think this is a chance that the Free Software community doesn't get every day. Let's hope some other people like bare iron programming in small embedded systems can get excited and we can create a FOSS COS. It doesn't have to be something serious. Something quite simple would be sufficient for the beginning. I'm not thinking of EAL4+ certification, multiple channels and public key crypto. SIM/USIM cards are simple, they just require a bit of filesystem read/write operations plus authentication. And luckily, SIM toolkit development doesn't have to be done in Java this way, either ;)