Blog of Alexander Mamchenkov http://alex.mamchenkov.net ... mammoth cave ... Mon, 26 Jan 2015 09:00:23 +0000 en-US hourly 1 http://wordpress.org/?v=4.1 Blog of Alexander Mamchenkov http://0.gravatar.com/avatar/28fa14800d5183df0b0e73cf9c22b5ee.png?s=48 http://alex.mamchenkov.net Instagram Daily Digest http://alex.mamchenkov.net/2015/01/26/instagram-digest-20150126/ http://alex.mamchenkov.net/2015/01/26/instagram-digest-20150126/#comments Mon, 26 Jan 2015 09:00:23 +0000 http://alex.mamchenkov.net/?p=5665 Спокойствие... Вот уж точно - весело Просыпашки в рестарашке) Курочка из духовки на обед. Жена умничка]]> http://alex.mamchenkov.net/2015/01/26/instagram-digest-20150126/feed/ 0 Plain SQL v.s. Zabbix API for text history items http://alex.mamchenkov.net/2015/01/18/plain-sql-v-s-zabbix-api-text-history-items/ http://alex.mamchenkov.net/2015/01/18/plain-sql-v-s-zabbix-api-text-history-items/#comments Sun, 18 Jan 2015 10:31:36 +0000 http://alex.mamchenkov.net/?p=5660 For one of the past tasks I have the following requirement: a number of hosts that are monitored with Zabbix has a item of history type that provide list of addresses in text format (one per line) and than I need to take all those lists and make one common list (table) where I would have in each row an address from the list, a hostname of the node where this address was last seen and timestamps.

Initially I added in zabbix the item to monitor on each node I needed, created a separate table in MySQL to hold the final list and then made a cron script that would do the following:

  • retrieve a list of nodes that have a given item by item key_ with Zabbix API
  • retrieve all items by itemid that I found out in a previous query with Zabbix API
  • for each item, retrieve the latest history with Zabbix API
  • for each history, split the text by new line to get addresses and then add each address with source host and timestamps in the final MySQL table, or update the timestamp and source in case address is already in the list

With total size of the list about 5K addresses, all of the above was taking around 4-5 minutes and was consuming a lot of CPU and memory on the server. As I was limited on server resources and wanted the list to be updated every minute, I decided to avoid using Zabbix API and try to the job with plain MySQL queries. As I am only interested in the latest history for each node, I recalled my own post that I did a while ago on SQL GROUP BY with subqueries. Checking zabbix SQL structure around and a bit of playing with queries, I ended up with the single request that will give me all I need:

SELECT * FROM (SELECT h.host,hi.id,hi.value\
FROM hosts AS h, items AS i, history_text AS hi\
WHERE i.hostid=h.hostid AND hi.itemid=i.itemid\
AND h.status<>3 AND i.key_ LIKE 'my_item_key%'\
AND hi.value <> '' ORDER BY hi.clock DESC) tmp_table\
GROUP BY host;

The sub-query will give me hostname, history entry id and history values from history_text for non template hosts (status <> 3), with non empty value for the item key_ I want order by time newest first and then the main query will take that list shrink it down to have only on entry per host which will be the newest one.

Now having this list, for each result raw, I can split the value by newline to extract all addresses and add them or update one by one in the final table. Here comes in another trick that I that I described in my old post here: since I need to update the source for already existed entries in the final table while adding anything that is not there, I run the insert with the following SQL statement, considering that the address field is a primary key and is unique:

INSERT INTO final_table (address,source)\
VALUES('$address','$source')\
ON DUPLICATE KEY UPDATE source='$host';

So whenever there is a conflict on address field, the source field is getting updated with the new value.

After changing Zabbix API queries to native SQL, the script runs few seconds and consumes almost nothing, as it is relying on MySQL engine to do most of the job, I MySQL can do it much better.

Finally, if there is no interest in history items after they were imported into final_table, it is possible to delete all raws for the items from history_text for a given key_ table with the following SQL query:

DELETE FROM history_text WHERE itemid IN\
(SELECT hi.id FROM items AS i, history_text AS hi\
WHERE hi.itemid=i.itemid AND i.key_ LIKE 'my_item_key%');

This is an alternative to relying on Zabbix housekeeper that will do the job, but a bit later. And if polling of the nodes for this item is pretty frequent and resulted values are pretty big – it will consume space in MySQL that we want to avoid.

]]>
http://alex.mamchenkov.net/2015/01/18/plain-sql-v-s-zabbix-api-text-history-items/feed/ 0
Instagram Daily Digest http://alex.mamchenkov.net/2015/01/14/instagram-digest-20150114/ http://alex.mamchenkov.net/2015/01/14/instagram-digest-20150114/#comments Wed, 14 Jan 2015 09:00:46 +0000 http://alex.mamchenkov.net/?p=5654 Небо супер]]> http://alex.mamchenkov.net/2015/01/14/instagram-digest-20150114/feed/ 0 Instagram Daily Digest http://alex.mamchenkov.net/2015/01/02/instagram-digest-20150102/ http://alex.mamchenkov.net/2015/01/02/instagram-digest-20150102/#comments Fri, 02 Jan 2015 09:05:35 +0000 http://alex.mamchenkov.net/?p=5648 Хорошо тусанули) Всех с новым годом! Happy new year to everyone! Падарочки)]]> http://alex.mamchenkov.net/2015/01/02/instagram-digest-20150102/feed/ 0 Instagram Daily Digest http://alex.mamchenkov.net/2014/12/31/instagram-digest-20141231/ http://alex.mamchenkov.net/2014/12/31/instagram-digest-20141231/#comments Wed, 31 Dec 2014 09:00:15 +0000 http://alex.mamchenkov.net/?p=5643 Домашний котенок Не читал почту 5 дней Травяной наркоман]]> http://alex.mamchenkov.net/2014/12/31/instagram-digest-20141231/feed/ 0 Instagram Daily Digest http://alex.mamchenkov.net/2014/12/10/instagram-digest-20141210/ http://alex.mamchenkov.net/2014/12/10/instagram-digest-20141210/#comments Wed, 10 Dec 2014 09:00:20 +0000 http://alex.mamchenkov.net/?p=5637 Плюша Люблю обедать дома. Жена чудо!]]> http://alex.mamchenkov.net/2014/12/10/instagram-digest-20141210/feed/ 0 Instagram Daily Digest http://alex.mamchenkov.net/2014/11/30/instagram-digest-20141130/ http://alex.mamchenkov.net/2014/11/30/instagram-digest-20141130/#comments Sun, 30 Nov 2014 09:00:04 +0000 http://alex.mamchenkov.net/?p=5629 Монстер компостер Вдруг откуда не возьмись]]> http://alex.mamchenkov.net/2014/11/30/instagram-digest-20141130/feed/ 0 VLC mosaic for multiple RSTP streams http://alex.mamchenkov.net/2014/11/27/vlc-mosaic-multiple-rstp-streams/ http://alex.mamchenkov.net/2014/11/27/vlc-mosaic-multiple-rstp-streams/#comments Thu, 27 Nov 2014 08:17:33 +0000 http://alex.mamchenkov.net/?p=5626 Recently had a task to open streams from 4 cameras over RSTP in a single windows using VLC. There is a bunch of howtos on net with relevant info and examples, but after trying many options non of them worked out of the box. Tweaking things here and there for a while I managed to come up with the working configuration, so I post it here for future reference and hopefully it will be helpful to anyone else.

My cameras stream dimensions is 1280×720 and as I want to fit 4 cameras on one screen, I will scale them in half to have screen size 1280×720 and each stream size 64×360.

First thing to do is to create some background image with exact size of the desired screen (1280×720 in my case) and save somewhere near (bg.jpg in my case).

Then we need to create a VLM config file (let it be cam.vlm.conf) to tell VLC about my streams and how to deal with them:

new channel1 broadcast enabled                                                       
setup channel1 input "rtsp://x.x.x.x:554/?user=foo&password=bar&channel=1&stream=0.sdp"
setup channel1 output #mosaic-bridge{id=1,height=360,width=640}

new channel2 broadcast enabled                                                       
setup channel2 input "rtsp://x.x.x.y:554/?user=foo&password=bar&channel=1&stream=0.sdp"
setup channel2 output #mosaic-bridge{id=2,height=360,width=640}

new channel3 broadcast enabled                                                       
setup channel3 input "rtsp://x.x.x.z:554/?user=foo&password=bar&channel=1&stream=0.sdp"
setup channel3 output #mosaic-bridge{id=3,height=360,width=640}

new channel4 broadcast enabled                                                       
setup channel4 input "rtsp://x.x.x.w:554/?user=foo&password=bar&channel=1&stream=0.sdp"
setup channel4 output #mosaic-bridge{id=4,height=360,width=640}

new mosaic broadcast enabled
setup mosaic input file:///home/user/Pictures/bg.jpg
setup mosaic option image-duration=-1
setup mosaic option image-fps=0
setup mosaic option mosaic-rows=2
setup mosaic option mosaic-cols=2
setup mosaic option mosaic-position=1
setup mosaic output #transcode{sfilter=mosaic,vcodec=mp4v,VB=8500,acodec=none,fps=25,scale=1}:display

control channel1 play
control channel2 play
control channel3 play
control channel4 play
control mosaic play

The input path to the camera streams as well as full path to the background image should be adjusted accordingly.

Somehow VLC doesn’t want to recognize mosaic-(height|width|order) parameters in the VLM file, so need to supply them inline as arguments when calling VLC. Now as we have VLM file ready, we can start the stream with the following command:

cvlc --vlm-conf /home/user/Desktop/cam.vlm.conf --mosaic-width 1280 --mosaic-order "1,2,3,4" --mosaic-height 720

Adjust the path to the VLM file accordingly as well as mosaic order or whatever else you want. For me all of the above worked out perfectly well and I can see all my 4 cameras in single window.

]]>
http://alex.mamchenkov.net/2014/11/27/vlc-mosaic-multiple-rstp-streams/feed/ 0
Instagram Daily Digest http://alex.mamchenkov.net/2014/11/24/instagram-digest-20141124/ http://alex.mamchenkov.net/2014/11/24/instagram-digest-20141124/#comments Mon, 24 Nov 2014 09:02:36 +0000 http://alex.mamchenkov.net/?p=5619 Ночной Лимассол ]]> http://alex.mamchenkov.net/2014/11/24/instagram-digest-20141124/feed/ 0 Instagram Daily Digest http://alex.mamchenkov.net/2014/11/20/instagram-digest-20141120/ http://alex.mamchenkov.net/2014/11/20/instagram-digest-20141120/#comments Thu, 20 Nov 2014 09:00:02 +0000 http://alex.mamchenkov.net/?p=5614 Ух ты мы улицу светим Photo of a selfie in progress)]]> http://alex.mamchenkov.net/2014/11/20/instagram-digest-20141120/feed/ 0