DIY - A parallel

Being picky in addition to being creative can lead to frustration.
Frustration that can be transformed into “something”.

DIY: “a term used by various communities that focus on people creating things for themselves” [1].
In the physical world, DIY often means to get one’s hands dirty.
In computer world, that means assembling software components.
In open source, that means doing a patch/pull request for an existing tool and/or creating a new tool altogether.
In all cases, DIY comes in when one needs to reduce the cost or/and to have something that fit perfectly into one’s environment.

I went through building a purpose built shelf and indoor flower hanger. Here is the reflection on the process and how it parallels to software (DIY).

Have a plan

It sounds obvious: the main requirement is to have a plan.
Going into DIY means you likely have a plan already, because you made the choice not to get it of the shelf.
The question is how much thinking went into the plan.

What is the goal

The goal is to fit the requirement into shape, shape that satify the end user(s).
In my case I needed some storage for books, DVDs, a TV stand.
I also needed that piece of furniture to serve as a standing desk.
It might do more, but cannot do less.
These are the minimum set of functions that the unit will serve.

What are the constraints

Cost, skills, time and access to resources needs to be factored in the plan.
I know I cannot afford a custom build shelf unit from a trades man.
I also know that I didn’t find anything fitting my need on the high street.
In order to get things done, I know that it shouldn’t require more than a week end to be completed.
I had the opportunity to get an experienced individual’s help.
I have a limited set of tools and a little appetite to hoard some more.

Constraint and context

Some constraints are equally limiting when writing software or building a shelf.
That said, some limits have greater visibility in the physical world.
I can see the exact space where the shelf will be, I control the runtime.
It is obvious that I need to take the radiator and the pipe going in and out of it into consideration.
In the physical world, things are payable upfront.
It is easier to reason with the cost of raw material than to think about running a LAMP stack on AWS.
I simply cannot buy rivets, screws, nails, epoxy adhesive, wood glue and tape and figure out later what I will need.
On the other hand, software makes some context explicit.
You cannot run some python app with a php interpretor.
You can have plastic garden chair with your freshly designe{r,d} table.

Execution

## Stick to the plan With the first hiccup in the plan, doubts will start to creep in.
The point of doubt is to try to make you change your mind, don’t. You might have to arrange for the fact that you only have 10 screws instead of 12.
You might want to hack things here and there for a non obvious skirting board angle.
You may realise that the end result will be less than optimal, but don’t forget that done > perfect

On tooling

A second obvious take on that DIY adventure is that tooling is important.
There is merit to start from scratch and use a little tooling.
There is also value in using the correct tooling, it’s not simply laziness.
If you have 100 screws and only 1 screwdriver, assembly will take a long time.
Then again, if you only have 100 screws, you don’t need the top of the range power tool. In software world, using top of the range tool is also payable later.
In software world, you pay in unused features and technical debt.

On experience

The last obvious realisation is that experience is worth all the tools in the world.
An experienced crafter delivers even with little to no tool.
The best power tool will let you build crooked furniture.

Build VS Buy

No word on buy vs build here.
There is nothing like the feeling of building something and getting it done.

[1] https://en.wikipedia.org/wiki/DIY_%28disambiguation%29

GitLab - How to replace text via the editor

If you often have to juggle between the GitLab text editor and the command line, you often find yourself in pain.
Until you realise you can use Crtl+h to open the replace function on the editor.
No more back and forth to do a quick sed.

GitLab-CI (Secure) variables

I have been using variables in GitLab-CI quite extensively and really like the way it nicely fits in GitLab.
In this blog post, I will illustrate the 2 type of variables that you can use, why you would use it and how.
The only requirement here is to have a GitLab account and a repository.
I will not cover the code nor the server side setup in details here since it is just an example.
It is to be noted that you can have private repository and CI on GitLab for free.

Getting started

To enable GitLab-CI, you will need a .gitlab-ci.yml file which is similar in fuction to the popular .travis.yml file.
It is good habit to lint your file before you commit them.

before_script:
  - curl -sSf https://static.rust-lang.org/rustup.sh -o rustup.sh
  - sh rustup.sh -y --disable-sudo
  
compile_test_ship:
  script:
    - rustc hello.rs
    - ./hello|grep -q -w "Hello World!"   
    -  $ curl -F $CI_BUILD_REF:hello $proto://$username:$password@$myserver:$portnumber/$url
  only:
    - master

I will not dive further on the .gitlab-ci.yml.
A quick look at the quick start guide will help you to get the big picture.
I always keep a tab open with the full documentation, just in case.

The variables

The interesting part of the file is the curl command arguments $CI_BUILD_REF:hello $proto://$username:$password@$myserver:$portnumber/$url.
Here we can see the two types of variables, the predefined variables:CI_BUILD_REF and the user-defined variables: proto:, username, myserver, portnumber, url.
The former are all caps and revolve around git and GitLab.
They give you access to build and project metadata.
The later can be defined in the yaml file, GitLab documentation refer to them as YAML-defined variables.
They can also be defined in the GUI via project settings -> variables -> add variable, GitLab documentation refer to them as Secure Variables.
You can overwrite variables, as you expect the Secure Variables take precedence over YAML-defined variables, both of which take precedence over Predefined Variables.

Predefine variables

Having access to predefine variable is really handy to automatically name things, especially version number with CI_BUILD_TAG.
In this example I use CI_BUILD_REF which contains the commit hash to name the binary on the server side.
As with any metadata, it is useful to print these to understand what is going on when you debug.

YAML-defined variables

These are the boring piece of data/configuration you might need for your project to run.
Your software use environment variable for configuration right ? If not, well it probably should.
The piece of data that are not boring like password and username have no place here.
Pieces of information that you do not want to share with the world should indeed not be here.

Secure Variables

Using secure variable for sensible information is the obvious use case.
If your project suddenly becomes visible to the world, you avoid an emergency password change.
In a similar note, if you get into the habit of separating concerns, you will/should not leak sensible data on your project.
The less obvious, but very valid use of the variable is to let forks have their own settings.
You can have identical code base, all the down to the CI, but have different setting across different version of the code.
One side effect of the feature is the granularity of access to the data.
When you need to give someone read/write access to you project, but not the details of your infrastructure beyond the CI.
There is likely more use cases, these are just the 3 that I encountered.

Secure not so secure

Giving your secrets to GitLab.com is a choice, a choice to trust their security team.
Or you can run your own security team, I use Sameer Naik’s docker image to run my own GitLab instance.
You can run your own GitLab instance and decide not to use secure variable.
I would argue that in the life span or your repo, you will regret such decision.
It might not be as secure as Travis CI and their encrypted variable.
Then again Secure Variables are easier to use than travis CI’s Encrypted Variables by far.
It depends what tradeoff do you prefer to live with.

Conclusion

I will never again (mistakenly or not) write any sensible information in plain text for my CI needs.
GitLab allows to hide secrets from sight and enable better habit.
How about your sensible information ?

Notes

An example of legal .gitlab-ci.yml that does not do what you expect.

before_script:
  - curl -sSf https://static.rust-lang.org/rustup.sh -o rustup.sh
  - sh rustup.sh -y --disable-sudo
  
stages:
  - build
  - test
  - deploy

compile_time:
  stage: build
  script:
    - rustc hello.rs
  only:
    - master

a_bit_of_testing:
  stage: test
  script:
    - ./hello|grep -q -w "Hello World!"
  only:
    - master

ship_it:
  stage: deploy
  script:
    - curl --data "@hello" $proto://$username:$password@$myserver:$portnumber/$url?dir=$CI_BUILD_REF
  only:
    - master

The rust source file:

fn main() {
    println!("Hello World!");
}

The output of the export command

$ export
declare -x BUNDLER_VERSION="1.11.2"
declare -x BUNDLE_APP_CONFIG="/usr/local/bundle"
declare -x BUNDLE_BIN="/usr/local/bundle/bin"
declare -x BUNDLE_PATH="/usr/local/bundle"
declare -x BUNDLE_SILENCE_ROOT_WARNING="1"
declare -x CI="true"
declare -x CI_BUILD_BEFORE_SHA="0000000000000000000000000000000000000000"
declare -x CI_BUILD_ID="678832"
declare -x CI_BUILD_NAME="compile_test_ship"
declare -x CI_BUILD_REF="dbc45e7c20bc860f98146f87e67053ce0cc7faec"
declare -x CI_BUILD_REF_NAME="master"
declare -x CI_BUILD_REPO="https://gitlab-ci-token:xxxxxx@gitlab.com/kyzh/hellors.git"
declare -x CI_BUILD_STAGE="test"
declare -x CI_PROJECT_DIR="/builds/kyzh/hellors"
declare -x CI_PROJECT_ID="844064"
declare -x CI_SERVER="yes"
declare -x CI_SERVER_NAME="GitLab CI"
declare -x CI_SERVER_REVISION=""
declare -x CI_SERVER_VERSION=""
declare -x GEM_HOME="/usr/local/bundle"
declare -x GITLAB_CI="true"
declare -x HOME="/root"
declare -x HOSTNAME="runner-ef64c3bf-project-844064-concurrent-0"
declare -x MYSQL_ALLOW_EMPTY_PASSWORD="1"
declare -x MYSQL_ENV_CI="true"
declare -x MYSQL_ENV_CI_BUILD_BEFORE_SHA="0000000000000000000000000000000000000000"
declare -x MYSQL_ENV_CI_BUILD_ID="678832"
declare -x MYSQL_ENV_CI_BUILD_NAME="compile_test_ship"
declare -x MYSQL_ENV_CI_BUILD_REF="dbc45e7c20bc860f98146f87e67053ce0cc7faec"
declare -x MYSQL_ENV_CI_BUILD_REF_NAME="master"
declare -x MYSQL_ENV_CI_BUILD_REPO="https://gitlab-ci-token:xxxxxx@gitlab.com/kyzh/hellors.git"
declare -x MYSQL_ENV_CI_BUILD_STAGE="test"
declare -x MYSQL_ENV_CI_PROJECT_DIR="/builds/kyzh/hellors"
declare -x MYSQL_ENV_CI_PROJECT_ID="844064"
declare -x MYSQL_ENV_CI_SERVER="yes"
declare -x MYSQL_ENV_CI_SERVER_NAME="GitLab CI"
declare -x MYSQL_ENV_CI_SERVER_REVISION=""
declare -x MYSQL_ENV_CI_SERVER_VERSION=""
declare -x MYSQL_ENV_GITLAB_CI="true"
declare -x MYSQL_ENV_MYSQL_ALLOW_EMPTY_PASSWORD="1"
declare -x MYSQL_ENV_MYSQL_MAJOR="5.7"
declare -x MYSQL_ENV_MYSQL_VERSION="5.7.10-1debian8"
declare -x MYSQL_NAME="/runner-ef64c3bf-project-844064-concurrent-0-build/mysql"
declare -x MYSQL_PORT="tcp://172.17.0.4:3306"
declare -x MYSQL_PORT_3306_TCP="tcp://172.17.0.4:3306"
declare -x MYSQL_PORT_3306_TCP_ADDR="172.17.0.4"
declare -x MYSQL_PORT_3306_TCP_PORT="3306"
declare -x MYSQL_PORT_3306_TCP_PROTO="tcp"
declare -x OLDPWD="/"
declare -x PATH="/usr/local/bundle/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
declare -x POSTGRES_ENV_CI="true"
declare -x POSTGRES_ENV_CI_BUILD_BEFORE_SHA="0000000000000000000000000000000000000000"
declare -x POSTGRES_ENV_CI_BUILD_ID="678832"
declare -x POSTGRES_ENV_CI_BUILD_NAME="compile_test_ship"
declare -x POSTGRES_ENV_CI_BUILD_REF="dbc45e7c20bc860f98146f87e67053ce0cc7faec"
declare -x POSTGRES_ENV_CI_BUILD_REF_NAME="master"
declare -x POSTGRES_ENV_CI_BUILD_REPO="https://gitlab-ci-token:xxxxxx@gitlab.com/kyzh/hellors.git"
declare -x POSTGRES_ENV_CI_BUILD_STAGE="test"
declare -x POSTGRES_ENV_CI_PROJECT_DIR="/builds/kyzh/hellors"
declare -x POSTGRES_ENV_CI_PROJECT_ID="844064"
declare -x POSTGRES_ENV_CI_SERVER="yes"
declare -x POSTGRES_ENV_CI_SERVER_NAME="GitLab CI"
declare -x POSTGRES_ENV_CI_SERVER_REVISION=""
declare -x POSTGRES_ENV_CI_SERVER_VERSION=""
declare -x POSTGRES_ENV_GITLAB_CI="true"
declare -x POSTGRES_ENV_LANG="en_US.utf8"
declare -x POSTGRES_ENV_MYSQL_ALLOW_EMPTY_PASSWORD="1"
declare -x POSTGRES_ENV_PGDATA="/var/lib/postgresql/data"
declare -x POSTGRES_ENV_PG_MAJOR="9.5"
declare -x POSTGRES_ENV_PG_VERSION="9.5.0-1.pgdg80+2"
declare -x POSTGRES_NAME="/runner-ef64c3bf-project-844064-concurrent-0-build/postgres"
declare -x POSTGRES_PORT="tcp://172.17.0.5:5432"
declare -x POSTGRES_PORT_5432_TCP="tcp://172.17.0.5:5432"
declare -x POSTGRES_PORT_5432_TCP_ADDR="172.17.0.5"
declare -x POSTGRES_PORT_5432_TCP_PORT="5432"
declare -x POSTGRES_PORT_5432_TCP_PROTO="tcp"
declare -x PWD="/builds/kyzh/hellors"
declare -x REDIS_ENV_CI="true"
declare -x REDIS_ENV_CI_BUILD_BEFORE_SHA="0000000000000000000000000000000000000000"
declare -x REDIS_ENV_CI_BUILD_ID="678832"
declare -x REDIS_ENV_CI_BUILD_NAME="compile_test_ship"
declare -x REDIS_ENV_CI_BUILD_REF="dbc45e7c20bc860f98146f87e67053ce0cc7faec"
declare -x REDIS_ENV_CI_BUILD_REF_NAME="master"
declare -x REDIS_ENV_CI_BUILD_REPO="https://gitlab-ci-token:xxxxxx@gitlab.com/kyzh/hellors.git"
declare -x REDIS_ENV_CI_BUILD_STAGE="test"
declare -x REDIS_ENV_CI_PROJECT_DIR="/builds/kyzh/hellors"
declare -x REDIS_ENV_CI_PROJECT_ID="844064"
declare -x REDIS_ENV_CI_SERVER="yes"
declare -x REDIS_ENV_CI_SERVER_NAME="GitLab CI"
declare -x REDIS_ENV_CI_SERVER_REVISION=""
declare -x REDIS_ENV_CI_SERVER_VERSION=""
declare -x REDIS_ENV_GITLAB_CI="true"
declare -x REDIS_ENV_MYSQL_ALLOW_EMPTY_PASSWORD="1"
declare -x REDIS_ENV_REDIS_DOWNLOAD_SHA1="e56b4b7e033ae8dbf311f9191cf6fdf3ae974d1c"
declare -x REDIS_ENV_REDIS_DOWNLOAD_URL="http://download.redis.io/releases/redis-3.0.7.tar.gz"
declare -x REDIS_ENV_REDIS_VERSION="3.0.7"
declare -x REDIS_NAME="/runner-ef64c3bf-project-844064-concurrent-0-build/redis"
declare -x REDIS_PORT="tcp://172.17.0.6:6379"
declare -x REDIS_PORT_6379_TCP="tcp://172.17.0.6:6379"
declare -x REDIS_PORT_6379_TCP_ADDR="172.17.0.6"
declare -x REDIS_PORT_6379_TCP_PORT="6379"
declare -x REDIS_PORT_6379_TCP_PROTO="tcp"
declare -x RUBYGEMS_VERSION="2.5.2"
declare -x RUBY_DOWNLOAD_SHA256="afd832b8d5ecb2e3e1477ec6a9408fdf9898ee73e4c5df17a2b2cb36bd1c355d"
declare -x RUBY_MAJOR="2.1"
declare -x RUBY_VERSION="2.1.8"
declare -x SHLVL="1"
declare -x myserver="posttestserver.com"
declare -x password="password"
declare -x portnumber="80"
declare -x proto="http"
declare -x url="post.php"
declare -x username="user"

GitLab, GitHub and Bitbucket - Adding an SSH key

I recently had to add my SSH to few services including GitLab, GitHub and Bitbucket.
As I added (some) SSH keys, I received some email about them.

A good email

What, in my opinion, makes a good email?
I have 3 (totaly arbitrary) criteria I found important in this case.

The key

Well, first off, I want to know what ssh key we are talking about.
GitLab provides the name/title for the key, which is a good start but not enough.
GitHub provides the name and fingerprint, which is better already, yet you have to use ssh-keygen -lf to verify.
Finally Bitbucket provides a the name, the type ssh-rsa, the first 17 and last 22 character o fthe key, along with the name.
All emails provided a link to go to the SSH cofiguration.

The unsubscribe button

I’m sure that you too have you fair share of email.
It is usually of good manner to provide a way to avoid further email.
GitLab had the touch of telling I could configuree it, yet not link.
GitHub didn’t even mentionned it.
Bitbucket at last had a link to the relevant part of the settings.
While it is possibly a bad idea to not have a notification, giving people the choice is more than welcome.

The email itself

If you use email for long enough, you will see some convention.
Mailing list for example often prefix the subject to add some clarity and help people sorting their email.
Another one convention is the sender field, where it make sense to have a sensible name.
GitLab doesn’t prefix the subject: SSH key was added to your account, but at least the sender field make sense with GitLab <gitlab@mg.gitlab.com>.
GitHub prefix the subject like so [GitHub] A new public key was added to your account and simple sender field GitHub <noreply@github.com>.
Bitbucket prefix the subject like so [Bitbucket] SSH key added to kyzh, but it looks like I sent my self an email with kyzh <notifications-noreply@bitbucket.org>.

Conclusion

Bitbucket is the most imformative and useful email.
I just don’t understand why using my name to send me an email, that’s confusing.
Finally, GitLab is the only where you can tangibly make a difference by contributing code to change its email.

On why Iceweasel didn't make a sound

Since I run debian 8, I had my browsing experience a little disminished.
I’m a big fan of not having flash installed.

I always had trouble with vimeo, and recently even youtube and other started to have issue.
Some video would play, some wouldn’t, but I never had sound.
I would usually get a local copy anyway (going back and forth within a stream tend to break).

Since I’m not really bothered, it was all good, well until today.
I wanted to use hubl.in with my coworker but I couldn’t hear them :/
Despite the fact that looking at them is cool, it is not realy helping to communicate.

I knew the problem would be with my setup.
I have 2 sounds cards, one being the default and one being the HDMI output.
The problem is that the first one to be detected and enabled is the HDMI one.

To get sound working for mplayer I had to do the follwoing.
Create a /etc/asound.conf with the following:

pcm.!default {
    type hw
    card 1
}

ctl.!default {
    type hw
    card 1
}

My default card is set to be the second one by order of apparition (0 being the first).
That way when application look for it, they don’t get the HDMI, but the card connected to my speaker instead.

If we look at aplay -l the order is still wrong

**** List of PLAYBACK Hardware Devices ****
card 0: HDMI [HDA ATI HDMI], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Generic [HD-Audio Generic], device 0: ALC269VB Analog [ALC269VB Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

With that in mind, I set myself to get the order right one layer above (or under if you prefer.)
The next step is to trick the sound module into re-ordering the cards.

I looked around a bit and while it all looked complicated, the solution is quite simple.
I edited (a then blank) /etc/modprobe.d/alsa-base.conf

options snd_hda_intel index=1

That way the first card detected is put to the index one.
Therefore the second card is able to take the index zero.
That way, I don’t need the /etc/asound.conf and aplay -l has the rigth order:

**** List of PLAYBACK Hardware Devices ****
card 0: Generic [HD-Audio Generic], device 0: ALC269VB Analog [ALC269VB Analog]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: HDMI [HDA ATI HDMI], device 3: HDMI 0 [HDMI 0]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

As usual, here is a film that I realy liked: blindness