Downgrade Raspberry Pi kernel

So I upgraded my DietPi kernel the other day and noticed there was no driver for Realtek 8188 driver for that kernel version. After playing around trying to compile the driver for the newest kernel I decided to downgrade the kernel. Thought it was difficult, but it’s quite easy. Only choose the kernel version you want to downgrade to from here, copy the hash and run rpi-update.

rpi-update  48cfa89779408ecd69db4eb793b846fb7fe40c4b

Hash above corresponds to kernel 4.4.11-v7+, with that kernel I was able to download the driver for my USB wifi using the script below:

#!/bin/bash
set -e

TOPIC_URL="http://www.raspberrypi.org/phpBB3/viewtopic.php?p=462982"

# Download and install rpi driver for 8188eu-based wifi dongles
# from MrEngman's dropbox.
#
# Version information is fetched from TOPIC_URL and appears as:
#
#   3.6.11+ #371 up to #520 inclusive    - 8188eu-20130209.tar.gz
#   3.6.11+ #524, #528, #532             - 8188eu-20130815.tar.gz
#   ...
# then is matched against local kernel release and version numbers
# to select proper driver tarball.  Kernel build number can be overriden
# with command line option -k, in case no exact match is found.

fetch_versions() {
	curl -s "$TOPIC_URL" \
	| sed 's:<code>\|</code>\|<br />:\n:g' \
	| sed 's:&nbsp;: :g ; s:gz.*:gz:' \
	| grep -E '^[0-9.]+.*tar\.gz'
}


case "$1" in
	-k|--kernel)
		build=$2
		;;
	-l|--list)
		fetch_versions
		exit 0
		;;
	-h|--help)
		echo "usage: `basename $0`" \
			"[-k|--kernel <kernel build>]" \
			"[-l|--list]"
		exit 0
		;;
	"")
		;; # proceed to install
	*)
		echo "unknown command: $1" >&2
		$0 --help
		exit 1
		;;
esac


kernel=$(uname -r)
build=${build:-$(uname -v | awk '{print $1}' | tr -d '#')}

if [ $kernel = "3.6.11+" ] && [ $build -gt 370 ] && [ $build -lt 521 ] ; then
	tarfile=8188eu-20130209.tar.gz
else
	tarfile=$(fetch_versions \
		| grep -e "^$kernel " \
		| grep -E "#$build[, ]" \
		| awk '{print $NF}')
fi

if [ ! "$tarfile" ] ; then
	echo "cannot match kernel: $kernel #$build"
	echo "please check news at $TOPIC_URL"
	echo "or try closest compatible version with -k <kernel build>"
	exit 1
fi

tmpdir=$(mktemp -d)
trap "\rm -rf $tmpdir" EXIT
cd $tmpdir

echo "downloading $tarfile (kernel $kernel #$build)"
curl -s https://dl.dropboxusercontent.com/u/80256631/$tarfile | tar xz

module_bin="8188eu.ko"
module_dir="/lib/modules/$kernel/kernel/drivers/net/wireless"
firmware_bin="rtl8188eufw.bin"
firmware_dir="/lib/firmware/rtlwifi"

if [ -f $firmware_bin ] ; then
	echo "installing firmware $firmware_bin"
	sudo install -p -m 644 $firmware_bin $firmware_dir
fi

echo "installing kernel module $module_bin"
sudo install -p -m 644 $module_bin $module_dir
sudo depmod -a
#sudo modprobe -r 8188eu || true # cannot currently be removed ("permanent")
sudo modprobe -i 8188eu
lsmod | grep -q 8188eu || echo "error: module not loaded"

As per latest update dl.dropboxusercontent.com is no longer valid and should be substituted by http://www.fars-robotics.net/, but dl.dropboxusercontent.com worked for me. Now my wifi is working.

uname -a; ifconfig wlan0; lsmod | grep 8188; lsusb
Linux DietPi 4.4.11-v7+ #886 SMP Thu May 19 15:20:49 BST 2016 armv7l GNU/Linux
wlan0     Link encap:Ethernet  HWaddr 00:e0:4c:81:89:01  
          inet addr:192.168.0.102  Bcast:192.168.0.255  Mask:255.255.255.0
          inet6 addr: fe80::2e0:4cff:fe81:8901/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:2262 errors:0 dropped:10 overruns:0 frame:0
          TX packets:1659 errors:0 dropped:3 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:463049 (452.1 KiB)  TX bytes:274451 (268.0 KiB)
8188eu                859474  0 
cfg80211              427855  1 8188eu
Bus 001 Device 004: ID 0bda:8179 Realtek Semiconductor Corp. 
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. SMSC9512/9514 Fast Ethernet Adapter
Bus 001 Device 002: ID 0424:9514 Standard Microsystems Corp. 
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

And below are my available kernels.

ls -ltr /lib/modules
total 28
drwxr-xr-x 3 root root 4096 Mar 18  2016 4.1.18-v7+
drwxr-xr-x 3 root root 4096 Nov 17 23:58 4.9.62+
drwxr-xr-x 3 root root 4096 Dec 30 12:36 4.9.35+
drwxr-xr-x 3 root root 4096 Dec 30 12:36 4.9.35-v7+
drwxr-xr-x 3 root root 4096 Dec 30 14:52 4.9.62-v7+
drwxr-xr-x 3 root root 4096 Dec 30 19:49 4.4.11+
drwxr-xr-x 3 root root 4096 Dec 30 20:03 4.4.11-v7+

Create a new table with Apache Hive

We are going to create a new table with Apache Hive from a previous one, populate it and then perform a UNION ALL of both tables. Below is the script that will create the new table.

-- Below script creates a new table
USE testdb;
-- show current tables
SHOW tables;
-- describe mytable2, table we will use to create mytable4
DESCRIBE mytable2;
-- create new table copying format from mytable2
CREATE TABLE mytable4 LIKE mytable2 ;

SHOW tables;
-- describe newly created table
DESCRIBE mytable4;
-- select content from newly created table
SELECT * FROM mytable4;

We proceed executing via hive in a linux shell.

hive    -f  create-new-table.hql

Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
OK
Time taken: 0.898 seconds
OK
mytable
mytable2
newtable3
Time taken: 0.206 seconds, Fetched: 3 row(s)
OK
id                  	int                 	                    
lname               	string              	                    
fname               	string              	                    
Time taken: 0.263 seconds, Fetched: 3 row(s)
OK
Time taken: 0.272 seconds
OK
mytable
mytable2
mytable4
newtable3
Time taken: 0.043 seconds, Fetched: 4 row(s)
OK
id                  	int                 	                    
lname               	string              	                    
fname               	string              	                    
Time taken: 0.166 seconds, Fetched: 3 row(s)
OK
Time taken: 0.666 seconds

Continue reading

Playing with functions in hive

Apache Hive has built in functions which can be listed with

SHOW FUNCTIONS;

to play with concat we will run the following script.

-- Use testdb
use testdb;
-- describe concat function
DESC FUNCTION concat;
-- describe table mytable2
DESC mytable2;
-- Perform select query uniting fname and lname
SELECT CONCAT(fname,' ',lname) FROM mytable2;

We can execute with beeline or hive. We will use beeline.

beeline -u jdbc:hive2://localhost:10000 -f Documents/concat.hql --verbose=false --showWarnings=false
scan complete in 8ms
Connecting to jdbc:hive2://localhost:10000
Connected to: Apache Hive (version 0.13.1-cdh5.2.0)
Driver: Hive JDBC (version 0.13.1-cdh5.2.0)
Transaction isolation: TRANSACTION_REPEATABLE_READ
0: jdbc:hive2://localhost:10000> use testdb;
No rows affected (0.104 seconds)
0: jdbc:hive2://localhost:10000> -- Describe concat function
0: jdbc:hive2://localhost:10000> DESC FUNCTION concat;
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                                          tab_name                                                                                           |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| concat(str1, str2, ... strN) - returns the concatenation of str1, str2, ... strN or concat(bin1, bin2, ... binN) - returns the concatenation of bytes in binary data  bin1, bin2, ... binN  |
+---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
1 row selected (0.162 seconds)
0: jdbc:hive2://localhost:10000> 
0: jdbc:hive2://localhost:10000> DESC mytable2;
+-----------+------------+----------+--+
| col_name  | data_type  | comment  |
+-----------+------------+----------+--+
| id        | int        |          |
| lname     | string     |          |
| fname     | string     |          |
+-----------+------------+----------+--+
3 rows selected (0.133 seconds)
0: jdbc:hive2://localhost:10000> 
0: jdbc:hive2://localhost:10000> SELECT CONCAT(fname,' ',lname) FROM mytable2;
+--------------------+--+
|        _c0         |
+--------------------+--+
| John Doe           |
| William Lancaster  |
| Burp Gentoo        |
+--------------------+--+
3 rows selected (18.848 seconds)
0: jdbc:hive2://localhost:10000> 
Closing: 0: jdbc:hive2://localhost:10000

We can also play with functions from inside hive cli as shown below with the sqrt function.

hive> DESC function sqrt;
OK
sqrt(x) - returns the square root of x
Time taken: 0.018 seconds, Fetched: 1 row(s)
hive> SELECT SQRT(64);        
Total jobs = 1
Launching Job 1 out of 1
Number of reduce tasks is set to 0 since there's no reduce operator
Starting Job = job_201710131004_0373, Tracking URL = http://localhost:50030/jobdetails.jsp?jobid=job_201710131004_0373
Kill Command = /usr/lib/hadoop/bin/hadoop job  -kill job_201710131004_0373
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 0
2017-12-06 12:33:16,450 Stage-1 map = 0%,  reduce = 0%
2017-12-06 12:33:23,476 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 1.58 sec
2017-12-06 12:33:28,497 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 1.58 sec
MapReduce Total cumulative CPU time: 1 seconds 580 msec
Ended Job = job_201710131004_0373
MapReduce Jobs Launched: 
Stage-Stage-1: Map: 1   Cumulative CPU: 1.58 sec   HDFS Read: 273 HDFS Write: 4 SUCCESS
Total MapReduce CPU Time Spent: 1 seconds 580 msec
OK
8.0
Time taken: 18.894 seconds, Fetched: 1 row(s)
hive> 

More info about Hive functions here.

Run httpd with docker

So below is the script:

#!/bin/bash

echo "Running httpd with docker."

docker run   --rm -v "$PWD":/usr/local/apache2/htdocs  httpd

We use the following options:
-v, –volume list Bind mount a volume
–rm Automatically remove the container when it exits

Quite simple, right? Execution below.

 bash ~/docker/run_httpd.sh
Running httpd with docker.
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.6. Set the 'ServerName' directive globally to suppress this message
AH00558: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.6. Set the 'ServerName' directive globally to suppress this message
[Thu Aug 17 22:22:55.249981 2017] [mpm_event:notice] [pid 1:tid 140029488904064] AH00489: Apache/2.4.27 (Unix) configured -- resuming normal operations
[Thu Aug 17 22:22:55.250079 2017] [core:notice] [pid 1:tid 140029488904064] AH00094: Command line: 'httpd -D FOREGROUND'

And we proceed to test.

 curl   http://172.17.0.6
<HTML>
<HEAD>
First page
</HEAD>
<BODY>
Testing docker httpd


We are getting below index.html because we are mapping /tmp/httpd (current $PWD) to /usr/local/apache2/htdocs. In /tmp/httpd we created an example index.html as shown above.
More info here

Run python script with docker

So I started playing with docker and asked myself whether it would be possible to run a python script with docker. Well, answer is yes. Example of the script below.

#!/usr/bin/python

import sys
print "Running script!!"
print sys.version_info

Execution below:

docker run -it --rm --name pythonscript -v "$PWD":/usr/src/myapp -w /usr/src/myapp python:2 python script.py
Running script!!
sys.version_info(major=2, minor=7, micro=13, releaselevel='final', serial=0)

References:
Docker documentation