Musings tagged as linux

As a continuation of my previously posted Sennheiser GSX 1000 setup this is the followup which comes from a better understanding of pulseaudio and the device itself. First of all, the GSX 1000 exports three hardware devices. hw:1,0 and hw:1,1 as outputs as well as hw:1,0 as a microphone input. Regarding the outputs, hw:1,0 is actually the mono chat output that is mixed into the main 7.1 channel output by the GSX 1000. The offset from the main volume can be controlled with the small knob on the right side of the device. This makes it especially interesting for use with voice applications like mumble or teamspeak. Your friends are too loud or too low? Twist the knob and adjust on the fly. hw:1,1 is the main 7.1 channel output which - at the moment - does not get detected by pulseaudio as a 7.1 channel device which is why I wrote about the workaround in the previous post.

This new and updated solution should ease usability and possibly make this a copy and paste effort not depending on your system config. This new approach leverages udev in order to catch you plugging in your device and a special udev rule then sets an environmental variable to tell pulseaudio to use a different configuration file for the card. Also it sets a unique id so it is easier to reference the card in case you have more than one soundcard connected.

First of all you need to create a file 91-pulseaudio-gsx1000.rules in the directory /lib/udev/rules.d/ or wherever your udev stores its rules. In this file you put the following:

SUBSYSTEM!="sound", GOTO="pulseaudio_end"
ACTION!="change", GOTO="pulseaudio_end"
KERNEL!="card*", GOTO="pulseaudio_end"

ATTRS{idVendor}=="1395", ATTRS{idProduct}=="005e", ENV{PULSE_PROFILE_SET}="sennheiser-gsx-1000.conf", ATTR{id}="GSX1000"


LABEL="pulseaudio_end"

1395 is the id for the vendor Sennheiser and 005e is the GSX 1000. After that we set the environment variable for pulseaudio and the id attribute of the soundcard. Now you need to create a second (and last, I promise!) file name sennheiser-gsx-1000.conf in your pulseaudio profile sets directory. For me this goes into /usr/share/pulseaudio/alsa-mixer/profile-sets/sennheiser-gsx-1000.conf:
[General]
auto-profiles = no

[Mapping analog-output-surround71]
description = main output
device-strings = hw:CARD=GSX1000,DEV=1
#device-strings = hw:%f,1
channel-map = front-left,front-right,rear-left,rear-right,front-center,lfe,side-left,side-right
paths-output = analog-output analog-output-lineout analog-output-speaker
priority = 2

[Mapping analog-output-chat]
description = chat output
device-strings = hw:CARD=GSX1000,DEV=0
#device-strings = hw:%f,0
channel-map = mono
paths-output = analog-output-headphones analog-output-headphones-2 analog-output-mono
priority = 1


[Mapping analog-input]
description = microphone input
device-strings = hw:CARD=GSX1000,DEV=0
#device-strings = hw:%f,0
channel-map = mono
paths-input = analog-input-front-mic analog-input-rear-mic analog-input-internal-mic analog-input-dock-mic analog-input analog-input-mic analog-input-linein analog-input-aux analog-input-video analog-input-tvtuner analog-input-fm analog-input-mic-line analog-input-headset-mic
priority = 2

# Combined output profile
[Profile output:analog-output-surround71+output:analog-output-chat+input:analog-input]
description = 7.1 Surround
output-mappings = analog-output-surround71 analog-output-chat
input-mappings = analog-input
priority = 88
skip-probe = yes

Now, all that is left is to reload udev, trigger the new ruleset and restart pulseaudio.

$> sudo udevadm control -R
$> sudo udevadm trigger
$> pulseaudio -k

After a couple of seconds pulseaudio should have restarted and you should have a nice and functional GSX 1000. Now, if someone has found a way to get software volume control working and how to keep the GSX from going Volume-Down all the time please tell me. :) Until then the temporary workaround for that problem remains unchanged and you can find it in the old article.

After a bit of a hiatus due to a new job engagement I am hopefully back on a more regular basis. For today with some instructions on getting a Sennheiser GSX1000 audio amplifier to run under Linux. This device is a 7.1 channel surround headphone amplifier with a lot of integrated extra functionality and a nice volume wheel. It connects via USB and there are a few steps to get it working correctly with pulseaudio. It does work right after plugging it in but only in mono mode which is not exactly what I bought this for. ;)

First of all let us get the ids of the sound card playback devices (sinks) via

$> aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: NVidia [HDA NVidia], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: NVidia [HDA NVidia], device 7: HDMI 1 [HDMI 1]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: NVidia [HDA NVidia], device 8: HDMI 2 [HDMI 2]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 0: NVidia [HDA NVidia], device 9: HDMI 3 [HDMI 3]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Audio [GSX 1000 Main Audio], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0
card 1: Audio [GSX 1000 Main Audio], device 1: USB Audio [USB Audio #1]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

In my case, card 0 defaults to the HDMI output of my NVidia GPU, let us ignore those entries. Card 1 however is the Sennheiser GSX 1000. I am actually not 100% sure on why it is listed twice with the same subdevices. From trial and error I managed to find out that the subdevice 1 is the correct one to drive the unit. From this information we know that the hardware address of the card for playpack (sink) is 1,1 (card 1 subdevice 1). Next we have to find out the recording device (source).

$> arecord -l
**** List of CAPTURE Hardware Devices ****
card 1: Audio [GSX 1000 Main Audio], device 0: USB Audio [USB Audio]
  Subdevices: 0/1
  Subdevice #0: subdevice #0

Our recording device (source) is at the hardware address 1,0 (card 1 subdevice 0 - as there is only one anyway). With this information we can tell pulseaudio how to correctly talk to our audio amplifier. Edit /etc/pulse/default.pa and under Load audio drivers statically we add

load-module module-alsa-sink device=hw:1,1 channels=8
load-module module-alsa-source device=hw:1,0

The first line tells pulseaudio that the playback device (sink) is located at hardware address 1,1 and has 8 (7+1) channels. The second line defines our recording device (source) as hardware address 1,0. At this point you can either restart your pulseaudio with pulseaudio -k or opt for a reboot if you still have not shaken your Windows heritage. ;) Voilla, it works!

The only problem left for me was that the turning of the volume ring gets detected by the kernel as volume down no matter the direction you turn the know. As such I went into the keyboard shortcut settings and disabled the multimedia hotkeys for volume up and volume down. I don’t mind losing that functionality as I want to set the master volume via the GSX1000 wheel anyway. After this small fix I can just use the hardware functionality of the volume wheel as supposed to and that leaves me one happy Alex ^^.

Update: I have found a more elegant way to handle this device. Check out the followup!

Lenovo Joins LVFS

Tue, Aug 7, 2018

Great news for a change! Lenovo has joined the Linux Vendor Firmware Service (LVFS). But wait, I hear you say, why should I care? Well, the LVFS is what makes firmware upgrades possible natively under Linux. In a nutshell this means in the not so distant future, it will be possible to update the UEFI bios of your Lenovo laptop natively via your local Linux installation. Awesomesauce!

Lenovo has been my goto laptop brand for years now. There are two main reasons for this:

  1. The hardware used in Lenovo laptops usually is well-supported by Linux. And why would you burden a nice piece of hardware with spyware crap like Windows?
  2. Lenovo publishes hardware maintenance manuals for their devices where you can read up on how to disassemble your laptop without breaking stuff to repair certain components yourself. Especially handy once you are out of warranty.

So now my favourite hardware manufacturer and my favourite OS move even closer together. This translates to very cool beans!

This is a followup to nginx RTMP Streaming With Simple Authentication.

Last time we covered a very basic setup with a hardcoded passkey. Multiple people have contacted me so far requesting an explanation on how to move towards a slightly more sophisticated authentication setup. Usually involving a php script to authenticate against. Maybe you want to use an existing mySQL or mariaDB database to set up users and channels? Fear not, this is not that complicated to start out with.

Server side configuration

Starting from the old example, we set up a basic rtmp section:

rtmp {
  server {
    listen 1935;
	ping 30s;
	notify_method get;
	  
	application stream {
	  live on;
	  on_publish http://yourdomain.com/rtmp_auth.php;
	  record off;
	}
  }
}

The on_publish command can point to any web address that you like. It could be supplied via the nginx server as well or you could use an apache2 instance for that. You can also use a completely different server if you wish. For now, we assume that there is a php script rtmp_auth.php which sits in the webroot of your webserver.

The above line will do the following:

  • as soon as someone tries to publish a stream to your domain, the nginx rtmp module will issue a HTTP POST request to the on_publish url.
  • nginx will supply the script with the get variable “name” and fill it with whatever comes directly after your initial stream url
  • it will also pass on any further GET style variables via the standard ?var1=value1&var2=value2 syntax.
  • it will wait for a HTTP return code which either tells it that everything is fine and streaming should commence (201) or that something went wront and it should drop the connection (404)

Suppose someone uses the following url to connect to your rtmp server:

rtmp://yourdomain.com/stream/john?psk=supersecret

nginx will then call your rtmp_auth.php script like this:

http://yourdomain.com/rtmp_auth.php?name=john&psk=supersecret

Inside of your php script you then have access to the $_POST array which holds your values and you can do whatever you want with them. In the following example we will use a php array $valid_users to hold a list of allowed users and passwords. Of course, you could instead connect to a database and query for the username and password. The interesting part is all in the if-statement which follows after that.

<?php
$username = $_POST["name"]; # in our current example, this will be 'john'
$password = $_POST["psk"]; # in our current example, this will be 'supersecret'

$valid_users = array("john" => "supersecret",
                     "winnie" => "thepooh",
					 "batman" => "nananananananana");

if ($valid_users[$username] == $password) {
  http_response_code(201); # return 201 "Created"
} else {
  http_response_code(404); # return 404 "Not Found"
}
?>

With this code, if the credentials check out, we return a 201 status code which tells nginx that whoever tries to connect is allowed to stream. If they do not, we issue a 404 and tell the client to get lost.

Client side configuration

Let us suppose your streaming client uses open broadcaster studio (OBS), which is a free and open source streaming utility which works with pretty much all major streaming sites and can also be configured to a custom site.

In OBS, you set the stream type to Custom Streaming Server and as the url you would use rtmp://yourdomain.com/stream/. As the stream key you would set john?psk=supersecret or any other username / password combination. If you want to supply more information you can supply more GET style variables via appending &var1=value1&var2=value2 and so on. Anything you write before the ? in the stream key field will end up in the variable $_POST["name"] inside your php script.

Sadly, SSL support for RTMP and also for the internal on_publishrequest is still somewhat lacking. So keep in mind that all of this stuff is plaintext authentication. You might not want to use a username / password combo directly but rather a streaming hash that you can allocate via a database and which has to be requested by the user beforehand. At least this way, if someone captures the data, they do not know the login credentials of your users.

You could for example generate a 64 character hash, put it into an SQL table and assign it to a user of your site. Then the user just enters this hash in OBS into the stream key field. Your php script will then be called and the variable $_POST["name"] will hold this hash. After that you can connect to your database, check out whether the hash exists or not and maybe even set up some notification on your website that user john is now streaming. Just be sure to finish with a 201 or 404 code in order to let nginx-rtmp know what it should do about the connection attempt.

In the end you can make things as complex as you want. You can use the same method for other nginx-rtmp directives such as: on_play, on_done, on_update and many more. Check them out at the nginx-rtmp wiki page.

Update: A kind reader has informed me that newer versions of the nginx rtmp plugin no longer use a GET but a POST request to call the URL you specify in the on_publish, on_play, etc directives. I have updated the code to reflect the changes.

Richard, a longtime maintainer for various open source projects, has offered Razor to do their dirty work for them and write firmware update software from scratch so Linux users that want to update their Razer devices without having to install Windows first can do so. All he asked for was some example code or at least the specifications so he could get started.

I offered to upstream any example code they could share under a free license, or to write the code from scratch given enough specifications to do so. This is something I’ve done for other vendors, and doesn’t take long as most vendor firmware updaters all do the same kind of thing; there are only so many ways to send a few kb of data to USB devices.

So how did Razer respond to the gift horse? I mean, they would be fools to turn down an offer of someone expanding their potential customer base for free, right?

I have discussed your offer with the dedicated team and we are thankful for your enthusiasm and for your good idea. I am afraid I have also to let you know that at this moment in time our support for software is only focused on Windows and Mac.

So don’t buy Razer hardware if you want to maintain them without a proprietary operating system. Personally I have never been a big fan of Razer as their desire to put everything into the cloud has taken one hillarious turn after the other. I’m sorry but my mouse settings do not need to be stored in the cloud and I do not want to create an account with your site just to use a piece of hardware I already paid for.

I’ve been looking for a simple way to utilize RTMP streaming with OBS Studio without having to resort to bloated websites like twitch. RTMP streams you can just play using your media player of choice (usually), for example VLC. In addition this can be useful to collaborate with colleagues, as you can stream not just a single window but also your whole desktop. nginx has access to a RTMP module (surprisingly called nginx-rtmp-module ^^) which they say does not do authentication but just the streaming. However, it features certain event calls like on_publish or on_play. After digging into it for a bit there is actually a super easy way to do a very simple authentication scheme which could be extended by any script you like.

All you need is first the RTMP block in your nginx config file:

rtmp {
  server {
    listen 1935;
	ping 30s;
	notify_method get;
			  
	application stream {
	  live on;
      on_publish http://localhost[:port]/auth;
      on_play http://localhost[:port]/auth;
	  record off;
	}
  }
}

This just sets up the stream and then forces to trigger a certain url on_publish (i.e. if someone wants to stream to the server) and on_play (i.e. someone trying to play back the stream). Now, the stream or play request will only be accepted if the url given returns a HTTP 2xx status code, else the connection will be dropped.

We can work this to our advantage and just set up a quick and dirty server in the http section of nginx that directly checks for a given secret:

server {
  listen <port>;
  location /auth {
  if ($arg_psk = 'totallysecretpassword') {
    return 201;
  }
  return 404;
}

You could also implement multiple urls with different secrets for streaming and playing or you could let a script of your choosing answer the url request and check for a username and a password or id in a database. Once all is done you can access the stream via rtmp://host.tld/stream/user?psk=totallysecretpassword. Of course this sends your password via plaintext so you should take precautions and if your software supports it use rtmp via ssl/TLS.

UPDATE: there is now a followup article on how to realize a slightly more sophisticated authentication setup

In order to get the new site up and running and switching to Hugo a few things were missing.

Pagination

I did not want a Next and Previous navigation as this gets really annoying with lots of pages. I wanted a list of pages, not too long through, and the ability to quickly traverse pages.

First we define the parameters for number of shown pages and at which position the active page is supposed to be located. This neatly describes the number of previous and following pages to generate. This goes into the config.toml site configuration:

[params]
	pgtrspan = 6
	pgtractive = 3

Then this goes into a partial template which you can include to show the paginator, lets call this pagenav.html:

<div id="page-bar">
  {{ $curpage := .Paginator.PageNumber }}
  {{ $lastpage := .Paginator.TotalPages }}

  <!-- if we have previous pages show quick rewind buttons -->
  {{ if .Paginator.HasPrev }}
  <div class="pagenumber">
   <a href="{{ .Paginator.First.URL }}"><span class="oi" data-glyph="caret-left" title="previous page" aria-hidden="true"></span><span class="oi" data-glyph="caret-left" title="previous page" aria-hidden="true"></span></a>
  </div>
  <div class="pagenumber">
   <a href="{{ .Paginator.Prev.URL }}"><span class="oi" data-glyph="caret-left" title="previous page" aria-hidden="true"></span></a>
  </div>
  {{ end }}

  <!-- get the width of the navpage in number of pages -->
  {{ $pgtrspan := $.Site.Params.pgtrspan }}

  <!-- get the index of the active page -->
  {{ $pgtractive := $.Site.Params.pgtractive }}

  
  {{ $lowerbound := sub $curpage (sub $pgtrspan (add $pgtractive 1)) }}
  {{ $upperbound := add $curpage (sub $pgtrspan $pgtractive) }}

  {{ if le $upperbound $pgtrspan }}
    {{ $.Scratch.Set "lowerbound" 1 }}
    {{ $.Scratch.Set "upperbound" $pgtrspan }}
  {{ else if gt $upperbound $lastpage }}
    {{ $.Scratch.Set "lowerbound" (add (sub $lastpage $pgtrspan) 1) }}
    {{ $.Scratch.Set "upperbound" $lastpage }}
  {{ else }}
    {{ $.Scratch.Set "lowerbound" $lowerbound }}
    {{ $.Scratch.Set "upperbound" $upperbound }}
  {{ end }}

  <!-- loop through paginator pages and only display stuff within the boundaries -->
  {{ range $id, $pager := .Paginator.Pagers }}
  {{ if and (ge $pager.PageNumber ($.Scratch.Get "lowerbound")) (le $pager.PageNumber ($.Scratch.Get "upperbound")) }}
  <div class="pagenumber">
    {{ if eq $curpage $pager.PageNumber }}
      {{ $pager.PageNumber }}
    {{ else }}
      <a href="{{ $pager.URL }}">{{ $pager.PageNumber }}</a>
    {{ end }}
  </div>
  {{ end }}
  {{ end }}

  <!-- if we have following pages show quick forward buttons -->
  {{ if .Paginator.HasNext }}
  <div class="pagenumber">
   <a href="{{ .Paginator.Next.URL }}"><span class="oi" data-glyph="caret-right" title="next page" aria-hidden="true"></span></a>
  </div>
  <div class="pagenumber">
   <a href="{{ .Paginator.Last.URL }}"><span class="oi" data-glyph="caret-right" title="previous page" aria-hidden="true"></span><span class="oi" data-glyph="caret-right" title="previous page" aria-hidden="true"></span></a>
  </div>
  {{ end }}
</div>

So, finally just put this below whatever paginator range you like:

{{ $paginator := .Paginate .Data.Pages }}
{{ range $paginator.Pages }}
<article>
    <h2><a href="{{ .Permalink }}">{{ .Title }}</a></h2>
    {{ .Content }}
</article>
{{ end }}
{{ partial "pagenav.html" . }}

And presto! Proper pagination bar.

Post Archive

Now the archive business is rather simple but took me a while to figure out as the documentation is really lacking in this regard. There are a couple of examples you can find online but none of them really did it the exact way I wanted it. Yearly archives are fairly easy to do but scrolling through a year worth of posts can be just as annoying if the post volume is high enough. So, ideally I want monthly archives but grouped by year. Leveraging this with Hugo’s taxonomy system is a tiny bit annoying but once it works it is perfectly serviceable.

First of all, add an archive taxonomy to your site’s config.toml

[taxonomies]
	archive = "archive"

Next, on all your posts, add the year and month as seperate taxonomy terms in the front matter

+++
title = "Wabbit"
date = 2017-12-09T16:59:48+01:00
archive = ["2017","2017-12"]
+++

So, the problem is how do we separate the YYYY terms from the YYYY-MM terms? Simple answer is, we don’t. When do we want the archive to start? That’s right, from the year and month we made the first post. When do we want it to end? How about at the latest post? So we create the termlisting taxonomy file as archive.terms.html which goes into one of the proper folders, in my case I use the layouts/taxonomy folder in the theme.

We then loop over all the years between the oldest and the newest post and after printing the list item for said year we loop through the terms. Only if a term matches the YYYY-MM format with YYYY matching the year we are currently indexing will it be printed. And since printing the term as is would be redundant as it contains the year again, we beautify it by use of dateFormat.

By the way, if someone knows a more elegant way in Hugo/Go-Template syntax to return true or false depending on whether a RegEx matches a string please drop me a line. The combination of len, findRE and a comparison operation was the shortest way I could think of with the limited toolset you get.

{{ partial "header.html" . }}

<section id="archive">
  <h1>Archive</h1>
  <ul>
    {{ $data := .Data }}
    {{ $firstyear := (index .Data.Pages.ByDate 1).Date.Format "2006" }}
    {{ $currentyear := now.Format "2006" }}

    {{ range $i,$year := (seq $currentyear $firstyear) }}
      <li><a href="/{{ $data.Plural }}/{{ $year }}">{{ $year }}</a><ul>
      {{ range $key, $value := $data.Terms.Alphabetical.Reverse }}
        {{ if gt (len (findRE (print "^" $year "-\\d\\d$") $value.Name)) 0 }}
          <li><a href="/{{ $data.Plural }}/{{ $value.Name }}">{{ dateFormat "January" (print $value.Name "-01") }}</a> ({{ $value.Count }})</li>
	{{ end }}
      {{ end }}
      </ul></li>
    {{ end }}
  </ul>
</section>

{{ partial "footer.html" . }}

Now, all that is left is to create archive.html which will be used to actually display the paginated list of articles for a year or for a month, depending on which gets selected by the viewer. We only check whether the term is in the YYYY or YYYY-MM format and output a header which is formatted accordingly. The rest is just simple iterating through the posts.

{{ partial "header.html" . }}

{{ $paginator := .Paginate (where .Data.Pages "Type" "musings") 10 }}
{{ $data := .Data }}

{{ if gt (len (findRE "\\d\\d\\d\\d-\\d\\d$" .Data.Term)) 0 }}
{{ $.Scratch.Set "heading" (print (dateFormat "January" (print $data.Term "-01")) " " (replaceRE "-\\d\\d$" "" $data.Term)) }}
{{ else }}
{{ $.Scratch.Set "heading" $data.Term }}
{{ end }}

<h1>Archive for {{ $.Scratch.Get "heading" }}</h1>
{{ range $paginator.Pages }}
<article>
  <h2><a href="{{ .Permalink }}">{{ .Title }}</a></h2>
    {{ .Content }}
</article>
{{ end }}
{{ partial "pagenav.html" . }}

{{ partial "footer.html" . }}

And we’re done.

1