Fixed CDN problem with KabelDeutschland streaming proxy

What’s new

As some of you already noted, there have been some issues with getting the channel-list from the KabelDeutschland API. They migrated to a new Content-Delivery-Network which does not support a get-header-Request anymore. This causes the proxy to generate invalid channel-links and your were unable to watch the streams.

This should be fixed by now. I committed the fix within the PHP and the GO version of the proxy and also compiled the binaries to a new version 0.1.3.

Please give me feedback if the problem is solved for you.

Binaries Download

Shared Folder at mega.co.nz (mirror at dropbox)

Sourcecode Repositories

PHP

https://github.com/edi-design/kd-streaming-proxy

GO

https://github.com/edi-design/kd-go

Code of the KabelDeutschland streaming proxy is available as open source

Why

A couple of times (stadt-bremerhaven.dedo-it-neat.com) I have been asked to open up the source code of my KabelDeuschland Streaming Proxy. There has been the PHP version available for quite some time on github (kd-streaming-proxy) but until now I was not able to open-source the version written in GO.

Now I got the time to clean up everything, wrote a lot of comments and put it on github for everyone to contribute.

https://github.com/edi-design/kd-go/

I hope there are a lot of developers out there using my proxy and are willing to help me with the further development.

About the code

The code is completely written in GO. It should be able to compile with any version above 1.3, but to be able to generate the cross-platform binaries you need version 1.5 of GO.

Dependencies

It only has one dependency to the gorilla mux router. You can install it by entering the following command.

Build the source

To build the source for various platforms, simply run the build-script provided under scripts/build.sh.
It relies on a proper configured go-environment with GOPATH and GOROOT set up.

By now, I only tested the build process on OS X but it should not be a problem to run the commands on Linux.

Configuration

Place a config.json next to the binary and fill it with the following content, including your KabelDeutschland credentials.

Run the binary

The easiest way, is to run the binary without any params. It searches automatically for the config.json next to the binary.

 

 

Version 0.1.2 of the KabelDeutschland streaming proxy

What’s new

This time, it is only a small update. I just added some improvements a couple of the users asked for. The most notable new feature is the ability to skip the certificate check. User Michael had the problem that his QNAP (TS-210) has no root certificates installed. This ended him up with an error message and prevented him from using the proxy.

I added a start-up trigger to disable the CA-Check. It is called -no-check-certificate.

 

Download

Shared Folder at mega.co.nz (mirror at dropbox)

Usage

To see a list of all triggers the proxy supports simply add -h.

If you want to see, if the disabling of the certificate check works, add -v -no-check-certificate. This disables the check and enables verbose mode to see a lot more informations.

 

Simplified KabelDeutschland streaming proxy

Review

A few days go i wrote about How to reverse engineer the KabelDeutschland API. Attached to this post there was a PHP script hosted on github to get your own streaming proxy running.

After testing and spreading the word with some people i came to the decision that the procedure was far too complicated. The need of a webserver and the large number of configuration options were unnecessary.

That’s why i build a single binary version, written in go.

Limitation

This tool does not remove the restrictions made by KabelDeutschland. You will need an internet and a TV contract with them. It only give you the ability to use their streams besides the iOS app.

Download

Binaries for all major platforms can be found at the shared folder.

Shared Folder at mega.co.nz (mirror at dropbox)

Select the binary suitable for your machine.

Tested environments

  • Macbook Pro Mid 2012, OSX 10.10.2
  • Synology DS215j (arm)
  • RaspberryPi (arm)
  • Windows XP 32bit

Please drop me a line if it works or not on your system.

Installation

Grab the config.json and edit it with your customer-service-center credentials. You can also change the port if you like.

On windows, double-click the executable file. On Linux/Mac you have to use your terminal, navigate to the folder you downloaded the binary and type the following command.

If you got a permission denied error, you have to set the executable bit for the binary

On all systems you will see the same output if everything went fine.

You will see the list of interfaces, the proxy listens to. All of them are ready to use to serve the channel playlist.

On certain Windows systems, it is not possible to display the interface list. The proxy will work nevertheless. Simply paste your known IP-Adress or one of your loopback addresses.

Installation on a Synology Diskstation

By now the easiest way to install the KabelDeutschland proxy is to download the arm-version of the binary and put it to your home-folder on the diskstation. Put the edited config.json next to it. Now you have to log into your nas  using ssh as your user and run the following command.

This creates a new screen, starts the proxy and immediately detaches the screen. That means the process runs in the background.

In order to be able to use the screen command, your box has to be bootsrapped. You can find the instructions to do this here: Bootrap the DS215j. Installation of screen will be done with this command as root.

You can connect to the proxy using the IP-address of your diskstation. http://[diskstation]:8787.

To stop the process, attach to the screen and press control+c. Attaching works using this command.

Predefined package

Synology has its own package manager and I’ll try to build a repository containing the proxy as a installable package. This will provide a way more easier solution. I will keep you updated.

Usage

Copy one of the links form the terminal output and paste it into your desired media player. It should load the playlist automatically. In VLC it looks like on the screenshot. You can now switch between the channels. The playlist contains links pointing to the proxy. This link handles the license generation, KabelDeutschland used to verify the stream.

Golang and reflections

Introduction

Reflection is the ability of a program to examine its own structure. In go we can use it for example to build dynamically accessable structs.

How-to use reflect

I will give you with two examples and a hopefully easy-to-understand explanation of them.

Simple usage

At hikewith.me we use a simple way of reflection to map dynamically specified keys to a set-up value. With this we can map database-columns to a given key and generate SQL queries according to the given input instead of predefining everything. In our database, field are named another way then in our backend code (go). For example the first-name of a user will be saved in the column first_name but the field in our User-struct is named FirstName. We need a way to map these connection dynamically for updating the user data and keeping our code simple.

Field-map struct

These struct type defines the connection between our used struct field and the columns found in our database table. FirstName, LastName etc. are the field names used in our User struct. The assigned value will be the table column. It will be loaded from a json-file (see JSON configuration).

User struct

This type stores the information about a user. If he changes some of his data we use this struct to update the dataset in our database.

JSON configuration

As already noted, we store the mapping between columns and keys in a JSON-file that will be loaded during the initial start of our backend.

Configuration loader

To load the JSON represented field configuration we use the gotamer-cfg-loader. You can get it using the following line.

The usage is really simple. Initialize the var to save the loaded configuration as type of your FieldMapping struct and afterwards simply load the file content into this struct.

Field-mapper

This is where the magic happens. The method will be called with the field name as param and returns the value of the field. First it reflects the list of all fields of the FieldMapping struct after this we can do an indirect reflect to get value of the field.

A more generic version

The following code is a more generic version of my code from above. It is not bound to a specific type but can be used with any kind of type to get a fields value. I will show how to use this function within the more advanced example.

Advanced usage

In this second example, i will show how to compare two structs of the same type with each other and get a diff. With this diff-struct we update our user data-set.

Compare struct

It is a simple key-value-store that will be used in a slice to collect all diffs.

Compare two structs of the same type

This function will be called from within a new-user-struct like this newuserdata.compareUpdate(olduserdata). The reflection will be used in the first place to get the number of fields in this struct. With this we can iterate over all fields and use the above defined getFieldByIndex function to get the value of each field and compare them to the corresponding value of the old-user-struct. The last step is to add the compare struct to our returned slice if there was a difference.