Making the Phone Vanish – Telephone Calls via an HT-503 and Raspberry Pi

Telephone, HT-503, and Raspberry Pi

Having spent two days fighting with my HT-503 box and telephones, I have achieved what I wanted. Two days ago I knew practically nothing about SIP, VoIP, Asterisk, and PSTN bridges. Now I can make landline calls to and from my computers.

I am now terrified that I will need to reconfigure my phones in the future and will forget everything I have struggled to achieve. Thus I am writing this for you, dear future me – and for you dear reader who has stumbled upon this hoping for answers.

Avoiding a Phone

We have a single landline connected to our house. This line is provided by our telephone provider and offers inclusive calls almost anywhere in the world. We use our landline to phone friends and family without worrying about call time or cost; everything is included in the monthly fixed fee.

We have a home network and a disliking of modern domestic landline phones. Home phones today pale in comparison to the cheapest mobile phones; so we decided to see if we could avoid them entirely.

Office phones are capable enough but ugly and fairly expensive. Researching office phones lured me into the world of Power over Ethernet (PoE) phones and Session Initiation Protocol (SIP) connected phones.

Ideally, we wanted to avoid needing a physical phone. A physical phone would mean having a single location to keep the phone in the house. It would be another item plugged into a wall and it would mean dedicating a prized network socket to a single purpose item, a phone.

Maybe we could use our computers as phones? They have speakers and microphones. Why not also use our trusted iPod Touch as a phone? Surely this was possible.

This led me down the path of SIP, Asterisk, and ATAs.

HT-503 – Bridging the Landline and Network

For the same cost as an office phone, or less, you can buy a Grandstream Networks HT-503 FXO/FXS Analog Telephone Adaptor (ATA). This small box of electronics can bridge the divide between a landline phone and a computer network.

Front of the Grandstream HT-503
Front of the Grandstream HT-503

When researching these bridging boxes, the trick is to look for an FXO port. This is the physical port you need to connect to an existing phone line provided by your telephone company.

Small boxes with FXO ports are far less common than boxes with the corresponding FXS ports. It is easy to become confused between the two similarly named ports.

So I bought a Grandstream HT-503 a couple of weeks ago for around fifty-five euros. The box is reasonably small, had the physical ports I needed, and has a feel of early twenty first century industrial design about it. The web interface is enough but it might be best to lower your expectations around ease of use.

Having checked the box powered on and seemed responsive, I took one look at the myriad of options, realised I was out of my depth, and put it aside for a couple of weeks.

What I Wanted

I wanted the following:

  • to be able to make phone calls from our computers through our landline connection;
  • to be able to answer incoming landline phone calls from our computers;
  • to have a working physical phone always connected for emergencies;
  • to be able to use software based phones, aka soft phones;
  • support for SIP devices in case we needed an office phone.

I did not care about:

  • being able to make outgoing VoIP calls to outside of the house;
  • being able to receive incoming VoIP calls from outside of the house;
  • being able to send or receive faxes.

What I Had – Equipment and Hardware

I had the following hardware:

  • a landline phone connection (RJ11 connection)
  • an ethernet network (RJ45 connections)
  • a Grandstream Networks HT-503 (RJ11 and RJ45 connections)
  • a Raspberry Pi 2 computer
  • an old home phone handset

What I Did – Setting up Phone Network

So this weekend, I set aside some time and threw myself into solving the home phone problem.

These are the steps I took to reach a working network based home phone system. The result works but is not the final configuration.

Set up Asterisk on Raspberry Pi with Debian

In my experiments I tried Asterisk for Raspberry Pi but found the FreePBX web interface too complex and overwhelming. I was more productive installing Asterisk on the Raspberry Pi’s lite image of Debian.

Debian Stretch Lite

Debian is a flavour of the open source and free Linux operating system. I chose Debian because it is the official operating system provided by the Raspberry Pi project; other Linux flavours exist and are probably just as good.

At the time of writing, the download version was:

Minimal image based on Debian Stretch
Version:September 2017
Release date:2017-09-07
Kernel version:4.9

Copy the image onto your Raspberry Pi’s microSD card, see installing images. On macOS:

  • Format the microSD card as FAT using Disk Image;
  • Note disk id of mounted card, in my case this was disk8;
  • Copy the image onto card using the Terminal command, using the appropriate disk id:
    sudo dd bs=1m if=2017-09-07-raspbian-stretch-lite.img of=/dev/rdisk8 conv=sync
    
  • Enable ssh access, see Remote Access > Secure Shell:
    touch /Volumes/boot/ssh
    
  • Place the microSD card into the Raspberry Pi and power it on.

Set up the Raspberry Pi

  • ssh into Raspberry Pi as the default user pi with default password of raspberry; find the IP address of the device via the switch or DHCP server.
  • Change the default password:
    passwd
    
  • Reclaim any spare microSD card space using raspi-config : Advanced > Expand Filesystem > Reboot:
    sudo raspi-config
    
  • Update Debian’s core software:
    sudo apt-get update
    sudo apt-get upgrade
    
  • Install ntp (Network Time Protocol) for accurate system time:
    sudo apt-get install ntp
    sudo /etc/init.d/ntp restart
    

Asterisk on a Raspberry Pi

Asterisk is an open source and free software based Private Branch Exchange (PBX). It is Asterisk that will handle the computers on the network and the calls coming in from and going to the HT-503.

  • Install Asterisk on the Raspberry Pi:
    sudo apt-get install asterisk
    

All the server software we need is now installed and ready to set up.

I started simply with Asterisk and have not gone much further than getting two extensions up and running with the landline. I followed the examples at A simple PBX system in The Asterisk Book to achieve this.

Asterisk is huge and complex. I deferred to the defaults in almost every case. I put aside the originals and started with blank files for the sip.conf and extensions.conf files:

cd /etc/asterisk
sudo mv sip.conf sip.conf-orig
sudo mv extensions.conf extensions.conf-orig
sudo touch sip.conf
sudo touch extensions.conf

These commands change into the folder, cd, containing all the Asterisk configuration files. Then the two key configuration files are renamed, mv, to remain safely available but out of the way. Two blank files are then created, touch, to take their place.

Within /etc/asterisk/sip.conf I created four extensions:

  • 2000
  • 2001
  • ht503fxo
  • ht503fxs

An extension is effectively a phone within the telephone system. 2000 and 2001 are from the Asterisk examples I followed and dedicated to soft phones on our computers. You can dial 2000 on the house phone and the computer or device within the house will ring.

The two ht503fxo and ht503fxs are extensions for the HT-503 to use. Numerous configuration guides suggested that an extension (SIP account) for the HT-503 was not required, or more strongly, must not be configured. I did not find this to be the case. Doug Crompton’s Replacing a Sipura SPA-3000
with a Grandstream HT-503
provided the first hint that this advice was misguided.

The complete /etc/asterisk/sip.conf file:

[general]
port = 5060
binaddr = 0.0.0.0
context = others

[2000]
type=friend
context=my-phones
secret=1234
host=dynamic

[2001]
type=friend
context=my-phones
secret=1234
host=dynamic

[ht503fxo]
type=peer
username=ht503fxo
secret=something
canreinvite=no
insecure=very
host=dynamic
nat=no
port=5062
disallow=all
allow=alaw
allow=ulaw
dtmf=rfc2833
qualify=yes
context=my-phones

[ht503fxs]
type=friend
username=ht503fxs
secret=something
host=dynamic
insecure=very
nat=no
dtmf=rfc2833
port=5060
canreinvite=no
disallow=all
allow=alaw
allow=ulaw
qualify=yes
context=my-phones

The complete /etc/asterisk/extensions.conf file:

[others]

[my-phones]
exten => 2000,1,Dial(SIP/2000,20)
exten => 2001,1,Dial(SIP/2001,20)

exten => _XXXXX.,1,Dial(SIP/ht503fxo,60,D(w${EXTEN}))
exten => _XXXXX.,n,Hangup()

The extensions are trivial in our case. The two internal extensions, 2000 and 2001, map to SIP accounts.

The final extension is a pattern _XXXXX. that matches any 6 or more digits. Matched calls are then passed to the HT-503 via SIP. The format of the Dial(SIP/ht503fxo,60,D(w${EXTEN}) is interesting and differs from some configuration guides. The line says, dial the ht503fxo, and then after connecting, dials ${EXTEN}. The w waits a couple of seconds.

I saw guides suggesting the format Dial(SIP/ht503fxo/${EXTEN}) but this did not immediately work for the HT-503. I suspect more experiments would reveal a better approach.

This Asterisk configuration is simplistic and likely needs hardening for security. In particular, I believe the context of the HT-503 extensions need attention. I also suspect some of the HT-503 key value pairs can be removed. For now though, it works.

Asterisk Hints

Use sudo asterisk -rvvvvvc to connect to a running instance of Asterisk. The extra verbosity is essential for debugging.

Set up the HT-503 for Asterisk

With the Asterisk server set up and running, I turned my attention to the HT-503. Discovering the appropriate settings for this box has been time consuming.

Back of the Grandstream HT-503
Back of the Grandstream HT-503

Plug in:

  • Plug nothing into the LAN port
  • Ethernet cable into the WAN port
  • Power into DC12V
  • Physical telephone into Phone port
  • Telephone line from wall into Line port

HT-503 Network Settings

Connect the HT-503 to the switch via the WAN port. Then use the physical phone connected to the HT-503 to enable WAN access to the web interface:

  • Dial **** (menu) > 12 (WAN access) > 9 (toggle to enable)

Reboot the HT-503:

  • Dial **** (menu) > 99 (reboot menu) > 9 (reboot)

My network is simple. The HT-503 is plugged into a switch. The network already has a DHCP server handing out IP addresses. The HT-503’s support for both local and wide area networks is not needed; let’s turn that off.

Basic Settings
  • Telnet Server: No
  • Device Mode: Bridge
  • WAN side HTTP/Telnet access: Yes

HT-503 Country Specific Settings

A telephone line’s behaviour is country specific. That means the following section is only relevant as a guide to what needs change but not what to change the values to.

My HT-503 is in France connected to a French landline. I need to change the box to behave appropriately for France.

Basic Settings

Time Zone: GMT+01:00

Advanced Settings
  • Dial Tone: f1=440@-10,f2=0@-10,c=0/0;
  • Ringback Tone: f1=440@-10,f2=0@-10,c=150/300;
  • Busy Tone: f1=440@-10,f2=0@-10,c=50/50;
  • Reorder Tone: f1=440@-10,f2=0@-10,c=50/50;

These lines tell the HT-503 what to expect for dial tones in France. I found the Interface Technical Specifications for France Telecom and Operational Bulletin No. 781 (1.II.2003) and Annexed List: Various tones used in national networkshelpful in crafting these. I am unsure about the volumes but -10 seems to work.

FXS Port
  • SLIC Setting: France
FXO Port
  • FXO Termination > PSTN Disconnect Tone: f1=440@-30,f2=440@-30,c=50/50;
  • FXO Termination > Country-based: France

HT-503 Asterisk Settings

With the network and country specific settings applied, the HT-503 is ready to connect to the Asterisk server.

Basic Settings
  • Unconditional Call Forward to VOIP: 2000 / ip.address.raspberry.pi / 5060

This setting tells the HT-503 to forward all incoming calls from the landline (PSTN) to the specified SIP account. In this case the account is for extension 2000 on our Asterisk server. This means any phone connected to the Asterisk server will ring when someone phones your home landline.

FXO Port
  • FXO Termination > Number of Rings: 2
  • FXO Termination > Impedance-based: Yes
  • FXO Termination > PSTN Ring Thru FXS: No
  • Channel Dialing > Stage Method (1/2): 1

This reduces the number of rings before the HT-503 picks up incoming calls. Two rings is enough for caller identification information to be collected by the HT-503 and passed onto the Asterisk server. You can reduce the number of rings to zero but you may loose Caller ID support.

By the time your computer’s soft phone rings, the caller will have heard two or three rings. This is because the HT-503 consumes the first two rings and only then starts sending the call to the network.

I am not convinced Impedance-based is needed.

The PSTN Ring Thru FXS determines whether a physical phone connected to the HT-503 rings when an incoming call arrives. I want only the computers to make a sound when calls come in, so I have disabled the physical phone from ringing.

Stage Method is critical. This setting determines how the HT-503 expects external numbers to be dialled. By default the HT-503 is configured to expect a two stage dialling process. The first stage is to dial the registered SIP account, ht503fxo. The caller will then hear the external dial tone. Only then is the external number called and this is referred to as stage 2.

We want to be able to dial our external number in one step. Our soft phone should only need to dial the external number. It is then the responsiblity of Asterisk and the HT-503 to cooperate. With Stage Method set to 1 this is what we get. Remember the Dial(SIP/ht503fxo,60,D(w${EXTEN})) in extensions.conf? This line connects to the HT-503 and immediately asks for the HT-503 to dial out the number we initially dialled.

If you wanted to be able to dial to just an external line, this is possible with:

exten => 2003,1,Dial(SIP/ht503fxo)

With this line in /etc/asterisk/extension.conf, a call to 2003 would end up hearing the dial tone of the external landline. From there you could dial a number on the keypad to connect elsewhere.

HT-503 SIP Settings

The SIP accounts on the HT-503 connect to the Asterisk server. The names of the accounts is deliberate the will hopefully ease pairing of the right port with the right account.

FXS Port
  • Account Active: Yes
  • Primary SIP Server: ip.address.raspberry.pi
  • Outbound Proxy: ip.address.raspberry.pi
  • SIP User ID: ht503fxs
  • Authenticate ID: ht503fxs
  • Authenticate Password: 1234
  • SIP Registration: Yes
FXO Port
  • Account Active: Yes
  • Primary SIP Server: ip.address.raspberry.pi
  • Outbound Proxy: ip.address.raspberry.pi
  • SIP User ID: ht503fxo
  • Authenticate ID: ht503fxo
  • Authenticate Password: 1234
  • SIP Registration: Yes

Reboot and Test

Congratulations on getting through all those settings. It is now time to reboot the HT-503, for luck, and start testing the phones work as desired.

Internet At Home Without a Phone Line

Our house move is nearly upon us and the telecom line has yet to be installed. Rather than panic, I set up a wireless network backed by a 4G mobile/cellular data connection.

The resulting connection is enough to work with but I still look forward to having a physical line. The laws of physics mean a physical connection will always outpace a wireless one.

Simple Approach

By chance I had a spare TP-Link TL-MR3420 bought for family but later no longer required. This surprisingly inexpensive box did exactly what I needed and with almost no fiddly set-up.

If you are looking for a fast, simple to connect solution, buy the TP-Link TL-MR3420 or whatever is the latest model. The low price of the TP-Link box is because you will need a separate USB 3G or 4G mobile stick. The USB stick is plugged into the TP-Link box and the box shares the mobile connection through a wireless network.

This means one mobile connection is shared across all the devices you have.

TP-Link TL-MR3420 + USB 4G modem = Easy Wireless Network

Want a more capable but more complex set up experience? Consider a RouterBOARD.

A Separate Identity

Bands of colour form in the sky during sunset.

We are putting together a web site to become our place for sharing photos and short films about the house build. The site will be a separate identity for what should be a well defined project. Intermingling house updates with general photos and news here on The Work Life simply felt messy.

The new site is called Love Allier and should start to see some content in the next week or so. I have already published three short films via a new YouTube channel. Those films have been challenging but enjoyable to create. Megan has written a few short posts to augment the films and to try to capture emotions that will be all too fleeting.

Sunset over our local park in Lyon.
Sunset over our local park in Lyon.

I want to be able to look back on the next two years as one event. Much like keeping a scrapbook or photo album dedicated to a specific period of your life, I hope that Love Allier will fill that role for us.

Whirlwind Through Our Schedules

With our recent purchase of land in France, our world has become fiendishly busy. The sudden switch from waiting to action has taken our breath away. In a few weeks we will settle back into a routine but for now it feels like a whirlwind is tearing through our schedules.

Two regional French TER train engines.
Two regional French TER train engines.

This morning we confirmed an appointment, booked our hire car, and only then discovered potential train strikes may make our travels impossible. Having gone to some effort to clear our schedules for that appointment, the possibility of re-scheduling and moving the bookings is frustrating.

This is an uncertainly I dislike but have to endure.

Dry Ink

Three sheep stare back.

The ink has dried on our land purchase. This morning the attestation arrived. The legal stage is done.

The attestation is a two page document bearing the stamp of a notaire. It sets out the details of the purchase and acts as proof the land transfer is complete. The attestation lets us demonstrate to the builder and infrastructure providers that the plot is ours.

The garden is going to be a challenge.
The garden is going to be a challenge.

Getting to this stage is more a relief than a celebration. We will celebrate the moment, but right now, I am simply pleased to have the legal side complete.

Wet Ink

The ink is still wet but we have signed. After close to a year of delays, today we signed to purchase a little plot of land in France.

The absolute purchase will not happen for a few days yet but both parties have signed and we have paid. Now the notaire needs to do the last chunk of paperwork and this long legal process will be finished.

A notaire plaque is displayed outside the office.
A notaire plaque is displayed outside the office.

When we first made the offer to buy, I never imagined the process to hand someone the money could take so long. There have been problems. We have had to fight and be persistent. At times it has been horrible. That period is ending.

Now we can look forward and get on with all that we postponed from last year. I am looking forward to this year being far more productive.

Delays and Page Shuffle

Trees and fog.

It has been a frustrating few weeks. We are working our way through the legal steps to secure our future home. Last week we learnt that there would be an additional delay.

The delay was not entirely unexpected. We saw it coming but others did not heed our warnings in time. After months of getting our side ready, even transferring funds into escrow, the meeting to sign was cancelled at the last moment.

A critical document was missing from the seller’s dossier. Without the document nothing can be signed and sealed.

Born of Annoyance

The funds have been returned and we await a new signing date.

Rather than sit impotently by, I channeled my energy into something I could control. I wrote and published a new application.

Shuffling Pages

A while ago Megan asked me how she could shuffle pages within a Portable Document Format (PDF) file. She had teaching resources that she wanted to reuse but the order and content of the pages was becoming predictable.

Had these been printed bingo or flash cards, she could have easily jumbled them up herself. But these were not physical cards. They were PDF documents on her tablet computer.

Over the following hour, I learnt that needing to shuffle pages within a document is rare enough to be difficult to do. Reordering or moving pages is possible but only one move at a time. OS X even includes a shuffle Automator action but it does not shuffle, it interleaves.

I put together a small command line tool to perform the shuffle. Since then, that tool has been good enough for Megan but not ideal. Having to use the Terminal.app is not a great experience.

With the changing of the signing date, it messed up my schedule. I found myself with a short gap in my schedule and too much personal distraction to focus on anything too demanding.

I looked through my list of possible future projects and settled to write Page Shuffle.

Page Shuffle's icon
Page Shuffle’s icon

Page Shuffle is not a complex application but it serves a purpose and does it well. I picked it because of that clear purpose and the few decisions that needed making during development.

Screenshot of Page Shuffle on OS X
Screenshot of Page Shuffle on OS X

To spice the application up a little I included an Automator action that actually shuffles PDF pages, rather than interleaves them. I also included a command line interface. I doubt either will see extensive use but they are available.

Channeled Creativity

Looking back this has been a response to not having control over something very important to me. Yet, being able to see oncoming problems with no influence to avoid them. The combination feels toxic.

Page Shuffle has been a silver lining. Not what I expected from this period, but a positive when all could be seen to be stagnating.

Hopefully this delay will not be for too much longer.

Imminent Changes

Sunset seen from Lyon, France

I am growing keenly aware that our world is about to change dramatically. Every aspect of our surroundings and lifestyle will alter when we move.

We have been here before. Having that experience only highlights the coming change.

When we moved from Australia to France, we had time to absorb the decision and make the move at roughly our own pace. We spent time with friends and family before getting on the plane. We packed our apartment up in Melbourne, knowing we would be unpacking into another apartment in Lyon. We were not hurried and we planned all we could.

It was a shift in country and culture, but ultimately a move from one city to another. The lifestyle shifted with the change in cultures but the surroundings were still of the cityscape variety. Cities tend to operate in the same way around the world. Any variation in the differences are highlighted and often celebrated; consider the praise a city can garner for their public transport or greenspaces.

Beautiful French balcony in Lyon
Beautiful French balcony in Lyon

Our pending move is different and likely more dramatic. We will remain in France – with all that entails – but our move is from city to countryside.

We have lived and loved living in city centres for the last decade. We have deliberately placed ourselves in the city centre. Central enough not to need a car, often not even needing any public transport.

For me, the coming move will be a return to more greenery. For Megan it will be her first time living in a smaller community. We both relish the impending change and challenge.

Where we hope for is not a rural retreat, it is not an isolated house, or vacated farm in the middle of vast tracts of countryside. We are not seeking the ex-communication and solitude that many moving to France’s countryside seek.

If all goes to plan, and we are not yet there, we will become part of a small community with neighbours, a few streets, and a church. A larger town with facilities is a walkable distance away. Our views will change from cityscape to countryside. Greenery and forest will replace the apartment blocks and buildings that surround us today.

Lyon's Notre-Dame de Fourvière
Lyon’s Notre-Dame de Fourvière

The move is well over a year away. The first sod of earth has not been shifted. Yet our weekends and most evenings are being spent planning this aspect and that of the change. There are countless details and decisions to be made.

The largest decisions have been made and are being slowly put into place. The single largest decision was choosing where to settle. Not the specific plot but the continent, the country, the region, and only then the community. That first grand decision, made almost unconsciously, has the greatest bearing. It influences every decision that follows.

It is easy to forget the biggest decisions. They come and go so quickly. The detail overwhelms and the broadest strokes are lost.

Now we are focusing on the small and sometimes tiny choices. We can plan for our next year and we can plan for our future life. But planning for the transition, for the setting up, for the putting in place, that is still vague, still unknown, and still ongoing.

Crafting Furniture

Slice of house model in SketchUp

There is something strangely cathartic about recreating furniture in SketchUp. Given good measurements, I can craft something recognisable in about ten to fifteen minutes.

As we wait for our planning permission to move through its various stages, I have been building up a model of our future home. The bulk of the building model is finished.

Now I am entertaining myself with little models of our existing furniture and, in a few places, items we are going to need.

A log burner modelled in SketchUp
A log burner modelled in SketchUp

Putting together these models reveals all kind of details I had not appreciated before. Dimensions, weights, and volumes all need consideration. You can not cheat with a three dimensional model.

SketchUp is surprisingly productive. It has taken a long while to understand its approach. The software’s demands on you, the operator, are not obvious but they are not too difficult to adopt. I have turned to YouTube tutorial videos many times. Little tips and tricks about typing dimensions, mass copying with keyboard adjustments, are wonderful but utterly hidden in the visual user interface.

As a tool I have found SketchUp impressively productive. The notion of pushing and pulling away material is effective. I initially tried Blender but it never felt enjoyable; a tool I likely need to invest more time into before I get the results I want out.

My growing collection of furniture looks surreal. Floating in a gray space. Ready to be copied and pasted into the house model.

Our growing collection of furniture recreated in SketchUp
Our growing collection of furniture recreated in SketchUp