mount and systemd

Had a task: double the size of a volume on amazon AWS EC2 instance. The process is yet manual and it is roughly as follows:

  • Create a new volume on AWS with double the size of the the old one
  • Attach it to the instance
  • Create partition and filesystem on the new volume
  • Mount the new volume somewhere next to the old volume mount point
  • Rsync data from the old volume to the new volume
  • Adjust /etc/fstab to point to the new volume for the corresponding mount point
  • Unmount both volumes
  • Mount the new volume to the old mount point
  • Detach the old volume from instance
  • Delete the old volume

All pretty simple and strait-forward. BUT! The new volume is not mounting to the old mount point, while mount command is silent about it!!!

Syslog gives a hint:

systemd: Unit var-whatever.mount is bound to inactive unit dev-xvdg1.device. Stopping, too.
lb1 systemd: Unmounting /var/whatever...
lb1 systemd: Unmounted /var/whatever.

That’s interesting. Why it is still bounded to inactive device (which I have already detached) and how I can unbound it?

Apparently all records in /etc/fstab are converted to systemd units and all mounting is (these ugly days) done via systemd. So when I changed /etc/fstab, the systemd didn’t update the the related unit and was still trying to mount the old device. To fix the problem you need to run:

systemctl daemon-reload

I am too old for this shit… Why are simple things getting more and more complicated (firewalld? ;-))

Robo.li and how to cook it

Introduction

One of the things I am heavily involved to is doing all sorts of infrastructure automation, and while ansible is a perfect tool for system administration automation, when it comes to more complex things like interacting with different APIs, and making all kind of more hardcore things, Robo.li (robo) is what can really save you.

I have started learning robo few weeks ago, but the problem is that they have pretty short documentation, and few basic examples, so it was not that easy at all and I am still in the process of finding best ways to go around, but I already have quite few things that I want to share here for anyone to use and for me have it saves in a safe place.

Ok, this one gonna be long enough and even probably a series of posts, but I have to share my experience.

While you can use single RoboFile.php with all commands you need in the project directory and call it via ./vendor/bin/robo, this is not a very comfortable approach for complex things, as this file get huge and messy very-very fast. For a proper use of robo, you should write a set of tasks, a set of command classes and then use all of it from your stand-alone robo app. For a sake of example, lets assume that all robo related commands and tasks are under src/Robo directory with Foo\Robo namespace prefix and you have a robo.php file with the actual app that you call.

To make it clear, here is a directory structure I have with some real tasks and commands already in place:

.
├── composer.json
├── composer.lock
├── .env
├── README.md
├── robo.php
└── src
    └── Robo
        ├── AbstractApiTask.php
        ├── AbstractCommand.php
        ├── AbstractTask.php
        ├── Command
        │   ├── BitbucketCommand.php
        │   ├── HipchatCommand.php
        │   └── RedmineCommand.php
        └── Task
            ├── Bitbucket
            │   ├── BranchRestrict.php
            │   ├── BranchUnrestrict.php
            │   ├── RepoCreate.php
            │   └── RepoDelete.php
            ├── Hipchat
            │   ├── RoomCreate.php
            │   └── RoomDelete.php
            └── Redmine
                └── ProjectCreate.php

(vendor directory is excluded on purpose to make tree clear)

Configuration

So now we have all in place and first thing first – we need to configure our app with a variety of different parameters. Robo does have Config implemented and configuration part is covered in a Robo as a Framework section of their site, but that mostly rely on storing your configuration in YML files, while I already have all configuration in a $config array and need to supply it to robo. To do that, we need to create a default robo container and pass our config to it, so in robo.php we have:

\Robo\Robo::createDefaultContainer(null, null, null, $config); 
$statusCode = \Robo\Robo::run(
    $_SERVER['argv'],
    $commandClasses,
    "Foo Robo",
    "v0.0.1"
);

Moreover, as we are passing the $commandClasses array with a list of the command classes, we want that part to be auto-generated as well, so above the code that creates a container we have:

$cmdPath = "src/Robo/Command";
$cmdNamespace = "\\Foo\\Robo\\Command";
$commandClasses = [];
foreach (glob("$cmdPath/*.php") as $file) {
    if (preg_match('/^.*\/(.*)\.php$/', $file, $matches) and class_exists($cmdNamespace . "\\" . $matches[1])) {
        $commandClasses []= $cmdNamespace . "\\" . $matches[1];
    }
}

And that’s pretty much all we need to have in robo.php apart of the code that fill the $config array, which is irrelevant to this blog post.

Task auto-loading

One thing that I found annoying in Extending robo how-to is the way they load tasks to commands. Basically for each collection of tasks you would have a loadTasks trait that describes each task and then you use it in the command class. While traits are questionable by nature, writing lots of same functions for a purpose of describing tasks is boring, time consuming and no way flexible. So instead of doing trait bullshit, we go with magic __call method in the AbstractCommand class that is extended by all actual tasks:

<?php
/**
 * Base command class for Foo Robo.li
 *
 * @see http://robo.li/
 */
namespace Foo\Robo;

use \Robo\Common\ConfigAwareTrait as configTrait;

abstract class AbstractCommand extends \Robo\Tasks
{
    use configTrait;

    /**
     * @var string $taskDir path to Tasks dir relative to our namespace
     */
    protected $taskDir = 'Task';

    /**
     * Magic __call that tries to find and execute a correct task based
     * on called method name that must start with 'task'
     *
     * @param string $method Method name that was called
     * @param array $args Arguments that were passed to the method
     *
     * @return 
     */
    public function __call($method, $args = null)
    {
        if (preg_match('/^task([A-Z]+.*?)([A-Z]+.*)$/', $method, $matches)) {
            $className = __NAMESPACE__ . "\\" . $this->taskDir . "\\" . $matches[1] . "\\" . $matches[2];
            if (!class_exists($className)) {
                throw new \RuntimeException("Failed to find class '$className' for '$method' task");
            }
            return $this->task($className, $args);
        }
        throw new \RuntimeException("Called to undefined method '$method' of '" . get_called_class() . "'");
    }
}

This class follows the naming convention and directory structure to auto-load tasks based on the method names and in out example the following code will work as expected from inside any command:

$this->taskHipchatRoomCreate()
     ->name("foobar")
     ->run();

$this->taskBitBucketRepoCreate()
     ->name("foobar.com")
     ->run();

So now we are focused only on writing tasks (example):

<?php

namespace Foo\Robo\Task\Hipchat;

use Robo\Result;
use Foo\Utility\Hash;

/**
 * Creates a HipChat room.
 *
 * 
 * <?php
 * $this->taskRoomCreate()
 * ->name('testroomname')
 * ->topic('This is a test room. Feel free to chat')
 * ->privacy('public')
 * ->run();
 * ?>
 * 
 */
class RoomCreate extends \Foo\Robo\AbstractApiTask
{
    /**
     * @var array data
     * @see https://www.hipchat.com/docs/apiv2/method/create_room
     */
    protected $data = [
        'name'                      => null,
        'privacy'                   => 'public',
        'delegate_admin_visibility' => null,
        'topic'                     => null,
        'guest_access'              => false
    ];

    /**
     * @var array requiredData
     * @see https://www.hipchat.com/docs/apiv2/method/create_room
     */
    protected $requiredData = [
        'name'
    ];

    /**
     * {@inheritdoc}
     */
    public function run()
    {
        
        $result = parent::run();
        if (!$result->wasSuccessful()) {
            return $result;
        }

        $this->printInfo("Creating {name} room", $this->data);

        try {
            $data = $this->api->post('room', $this->data);
        } catch (\Exception $e) {
            return Result::fromException($this, $e);
        }

        if (!$data || !is_array($data)) {
            return Result::error($this, "Failed to create room", $this->data);
        }

        // Because the create call returns only id, we'll make a new call to get
        // complete info about new room

        $this->printInfo("Retrieving {name} room info", $this->data['name']);
        
        try {
            $data = $this->api->get('room/' . $data['id']);
        } catch (\Exception $e) {
            return Result::fromException($this, $e);
        }

        if (!$data || !is_array($data)) {
            return Result::error($this, "Failed to retrieve room info", $this->data);
        }

        $data = array_filter(Hash::flatten($data));
        return Result::success($this, "Room successfully created", $data);
    }
}

and commands (example):

<?php

/**
 * Foo HipChat Robo.li commands file
 *
 * @see http://robo.li
 */

namespace Foo\Robo\Command;

use \Consolidation\OutputFormatters\StructuredData\PropertyList;
use \Foo\Robo\AbstractCommand;

class HipchatCommand extends AbstractCommand
{

    /**
     * Create a HipChat room
     *
     * @param string $name Room name
     * @param string $topic Room topic
     * @param string $privacy Room privacy
     *
     * @option string $format Output format (table, list, csv, json, xml)
     * @option string $fields Limit output to given fields, comma-separated
     *
     * @return PropertyList Room info
     *
     * @field-labels:
     *   id: id
     *   name: name
     *   topic: topic
     *   privacy: privacy
     *   xmpp_jid: xmpp_jid
     *   is_archived: is_archived
     *   is_guest_accessible: is_guest_accessible
     *   owner.id: owner_id
     *   owner.name: owner_name
     *   created: created
     *   last_active: last_active
     */
    public function hipchatRoomCreate($name, $topic = "", $privacy = 'public', $opts = ['format' => 'table', 'fields' => ''])
    {
        $result = $this->taskHipchatRoomCreate()
            ->name($name)
            ->topic($topic)
            ->privacy($privacy)
            ->run();

        return new PropertyList($result->getData());
    }

    /**
     * Delete HipChat room
     *
     * @param string $name Room name
     *
     * @option bool $force Force deletion
     *
     * @return bool true on success and false on failure
     */
    public function hipchatRoomDelete($name, $opts = ['force' => false ])
    {
        if (!$opts['force']) {
            $this->say("Won't delete anything unless you force me to");
            return false;
        }

        return $this->taskHipchatRoomDelete()
            ->name($name)
            ->run();
    }
}

I hope this post will be useful for someone (at least for me). There are few things in the examples above that are also tricks and not really standard to robo, including printInfo() method, data and requiredData properties, etc, but that will be covered later in another post.

BitBucket push surprise

Today I was very surprised and even scared by a response from pushing my changes to a repository hosted on BitBucket.

Not that I am against of people celebrating whatever they want, changing site logos and whatsoever, but this way too much, especially in such types of the tasks. When you do a lot of pushes all the time, you mind is getting used to recognise patterns in the response messages act accordingly, but this one screws the mind completely. To make it simplier: imaging the traffic lights would change colours from time to time to for similar occasions…

Google Chrome Fullscreen

Have a problem with Google Chrome not willing to exit fullscreen mode (by F11, or clicking “Exit full screen” bubble link) when run on a secondary monitor on my Fedora. Pretty annoying and the only way to restore to normal was to re-open chrome, but this is not always a good thing for me.

Today I finally figured out the way to exit fullscreen without a need to reopen Chrome:

  • press Ctrl+N to open new Chrome window (it will open on main monitor and not in fullscreen mode)
  • go to settings and select “Use system title bar and borders”
  • go back to the fullscreen window of Chrome that is on the secondary monitor and press F11 to get it out the fullscreen (this time it will work)
  • untick the “Use system title bar and borders” in settings to get back a normal look

A bit messy, but I hope one day this bug will be sorted out.

PrimeTel SIM card replacement

PayAsYouGoOk, as I had couple of post here and here with regards to issues I faced with PrimeTel and the fact that my wifi did a nice post in her blog concerning the issue of changing PrimeTel pay as you go SIM card from normal to mini, I will be short this time.

What I had:

  • CYTA SoEasy normal size SIM card for regular guests who come from abroad to me and want to use local phone.
  • PrimeTel pay-as-you-go normal size complimentary SIM card that I received from PrimeTel for my wife to try.
  • My wife’s Sony Xperia S that needs can accept a mini-SIM only.
  • My wife’s CYTA SoEasy mini size SIM card that was given to my wife free of charge in the first CYTA shop on our way when we go the Sony Xperia S mentioned above as a replacement for the normal size SIM card that my wife used in her old phone.
  • Verified fact by many hours of playing Ingress outdoor using my and my wife’s mobiles that PrimeTel 3G works way better than CYTA 3G.

What I thought of doing:

  • Replace the PrimeTel pay-as-you-go normal size complimentary SIM card (not activated, not removed from original plastic card container) with PrimeTel pay-as-you-go mini size SIM so that my wife can try it and enjoy good quality 3G
  • Keep both CYTA SoEasy SIM cards for guests and give them an option to use normal size or mini, depending on their mobile device, as well as use these cards as a backup personally if I need.

SIMWhat I end up with:

  • I was told by PrimeTel customer support in the PrimeTel shop that to replace a SIM card I need to pay 10 euro. This is completely weird, as the price of the actual PrimeTel pay-as-you go SIM card is 7.50 euro and that include 6 euro talk +5MB 3G. How come?
  • My wife said: “Fuck that! Now I will stay on CYTA SoEasy for sure and nothing gonna change my decision, as I am fed up with this way of doing things”.
  • I will replace CYTA SoEasy normal size SIM card that I currently have for guests to a mini SIM for free at the first CYTA shop I manage to pass by.
  • I will keep the PrimeTel pay-as-you-go normal size SIM card as an alternative for guests.
  • I will continue using PrimeTel post-paid SIM as I have now in my phone, since I am pretty satisfied how it works for time-being, but I am sure that my wife will never switch to PrimeTel after all of this.

It is really pity that small things that really matter are not well dealt with. You can have good quality products, but minor mistakes can spoil everything…