Tag Archives: linux

Primitive way with Folium

So I discovered Folium about two months ago and decided to map the primitive way with it. Coordinates data is retrieved from Strava gpx files and cleaned up leaving only latitude and longitude as below.

user@computer:$ head Camin_prim_stage1.csv
lat,lon
43.3111770,-5.6941620
43.3113360,-5.6943420
43.3114370,-5.6944600
43.3115000,-5.6945420
43.3116970,-5.6948090
43.3119110,-5.6950900
43.3122360,-5.6956830
43.3123220,-5.6958090
43.3126840,-5.6963740

Below is the python file we will use to retrieve data and create the map with the routes.

import folium
from pyspark.sql import SparkSession
from pyspark.sql.functions import col
spark = SparkSession.builder.master("local").getOrCreate()

# Change Spark loglevel
spark.sparkContext.setLogLevel('FATAL')

# Load the rides and ride_routes data from local instead of HDFS
position1 = spark.read.load("/home/user/Camin_prim_stage1.csv", format="csv", sep=",", inferSchema="true", header="true")
position2 = spark.read.load("/home/user/Camin_prim_stage2.csv", format="csv", sep=",", inferSchema="true", header="true")
position3 = spark.read.load("/home/user/Camin_prim_stage3.csv", format="csv", sep=",", inferSchema="true", header="true")

position = [position1, position2, position3]

m = folium.Map()
col=0
colArray=['red','blue','green']

# Check file was correctly loaded
for x in position:
# x.printSchema()
# x.show(2)

# Map position
coordinates = [[float(i.lat), float(i.lon)] for i in x.collect()]

# Make a Folium map
#m = folium.Map()
m.fit_bounds(coordinates, padding=(25, 25))
folium.PolyLine(locations=coordinates, weight=5, color=colArray[col]).add_to(m)
folium.Marker(coordinates[1], popup="Origin").add_to(m)
folium.Marker(coordinates[-1], popup="Destination").add_to(m)
col = col + 1
# Save to an html file
m.save('chamin_prim.html')

# Cleanup
spark.stop()

Continue reading

Docker in DietPi

So I was trying to install docker in my raspberry pi with DietPi and initially installed it via repo/aptitude, but that version is pretty old and not supported anymore.

user@computer:$ dpkg -l | grep dock
ii docker.io 1.3.3~dfsg1-2 armhf Linux container runtime

user@computer:$ sudo docker version
Client version: 1.3.3
Client API version: 1.15
Go version (client): go1.3.2
Git commit (client): d344625
OS/Arch (client): linux/arm
Server version: 1.3.3
Server API version: 1.15
Go version (server): go1.3.2
Git commit (server): d344625

Continue reading

GPS on Orange Pi 2G IOT

Recently I got myself a GPS module to play with such as this. Not that I knew much about but felt like something nice to play with. First thing first is connecting the GPS module to the GPIO in Orange Pi 2G. In my case I decided to connect the GPS to ttyS2, asides from VCC (2.8V) and GND. This correspond to pins 1, 6, 8 and 10.
Vcc -> pin 1
GND -> pin 6
TxD -> pin 8
RxD -> pin 10

Pin 1 would be the one just below a small white arrow to the left of the SD card. Below two pictures of GPS connected to Orange Pi GPIO pins.


Another important thing is to cross TxD and RxD cables, that means connecting GPS TxD to Orange Pi RxD and GPS RxD to Orange Pi TxD.
Now that the GPS module is connected to the Orange Pi power it up and log into it. We will now check if the GPS mocule is working, in order to do this we will connect via minicom.

user@computer:$ minicom -D /dev/ttyS2 -b 9600
Welcome to minicom 2.7

OPTIONS: I18n
Compiled on Apr 26 2017, 00:45:18.
Port /dev/ttyS2, 19:10:00

Press CTRL-A Z for help on special keys

$GPRMC,191013.00,A,4252.31265,N,7833.00897,W,0.180,,040518,,,A*64
$GPVTG,,T,,M,0.180,N,0.333,K,A*29
$GPGGA,191013.00,4252.31265,N,7833.00897,W,1,06,2.16,296.4,M,51.3,M,,*44
$GPGSA,A,3,22,16,10,08,26,27,,,,,,,3.81,2.16,3.14*04
$GPGSV,3,1,10,01,19,244,,08,60,313,24,10,49,079,28,16,36,164,36*74
$GPGSV,3,2,10,18,42,250,22,20,07,038,,21,06,061,,22,09,195,30*7A
$GPGSV,3,3,10,26,09,158,30,27,69,072,27*70
$GPGLL,4252.31265,N,7833.7897,W,191013.00,A,A*7C

Continue reading

Fixing fail2ban

I had installed fail2ban but had noticed it wasn’t working blocking ssh brute force attacks. Attacks such as below.

user@computer:$ grep sshd /var/log/auth.log | tail
Apr 29 08:06:17 sd-229337 sshd[20646]: pam_unix(sshd:auth): check pass; user unknown
Apr 29 08:06:17 sd-229337 sshd[20646]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=211-75-3-35.hinet-ip.hinet.net
Apr 29 08:06:18 sd-229337 sshd[20646]: Failed password for invalid user db2inst from 211.75.3.35 port 52724 ssh2
Apr 29 08:06:19 sd-229337 sshd[20646]: Connection closed by 211.75.3.35 [preauth]
Apr 29 08:18:21 sd-229337 sshd[20711]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=59-120-243-8.hinet-ip.hinet.net user=root
Apr 29 08:18:25 sd-229337 sshd[20711]: Failed password for root from 59.120.243.8 port 34312 ssh2
Apr 29 08:18:25 sd-229337 sshd[20711]: Connection closed by 59.120.243.8 [preauth]
Apr 29 08:19:14 sd-229337 sshd[20713]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=195-154-136-62.rev.poneytelecom.eu user=root
Apr 29 08:19:16 sd-229337 sshd[20713]: Failed password for root from 195.154.136.62 port 24329 ssh2
Apr 29 08:19:16 sd-229337 sshd[20713]: Connection closed by 195.154.136.62 [preauth]

In order to fix this we need to modify /etc/fail2ban/filter.d/common.local and modify bsd_syslog_verbose entry. Substitute __bsd_syslog_verbose = (<[^.]+\.[^.]+>) for __bsd_syslog_verbose = (<[^.]+ [^.]+>).

user@computer:$ grep bsd_syslog_verbose /etc/fail2ban/filter.d/common.local
#__bsd_syslog_verbose = (<[^.]+\.[^.]+>)
__bsd_syslog_verbose = (<[^.]+ [^.]+>)
__prefix_line = \s*%(__bsd_syslog_verbose)s?\s*(?:%(__hostname)s )?(?:%(__kernel_prefix)s )?(?:@vserver_\S+ )?%(__daemon_combs_re)s?\s%(__daemon_extra_re)s?\s*

Restart fail2ban and you shall now see IPs performing brute force attacks being blocked as below.

user@computer:$ tail -30 /var/log/fail2ban.log | grep actions
2018-04-29 18:43:19,835 fail2ban.actions[28271]: WARNING [ssh] Unban 163.172.159.119
2018-04-29 18:43:20,742 fail2ban.actions[28519]: INFO Set banTime = 1800
2018-04-29 18:43:20,936 fail2ban.actions[28519]: INFO Set banTime = 600
2018-04-29 18:43:59,119 fail2ban.actions[28519]: WARNING [ssh] Ban 171.244.27.195
2018-04-29 18:46:05,286 fail2ban.actions[28519]: WARNING [ssh] Ban 5.188.10.185
2018-04-29 19:13:59,938 fail2ban.actions[28519]: WARNING [ssh] Unban 171.244.27.195
2018-04-29 19:14:50,026 fail2ban.actions[28519]: WARNING [ssh] Ban 171.244.27.195
2018-04-29 19:15:35,102 fail2ban.actions[28519]: WARNING [ssh] Ban 159.65.10.166
2018-04-29 19:16:06,167 fail2ban.actions[28519]: WARNING [ssh] Unban 5.188.10.185
2018-04-29 19:44:50,740 fail2ban.actions[28519]: WARNING [ssh] Unban 171.244.27.195
2018-04-29 19:45:35,821 fail2ban.actions[28519]: WARNING [ssh] Unban 159.65.10.166
2018-04-29 19:45:38,858 fail2ban.actions[28519]: WARNING [ssh] Ban 171.244.27.195

But why is this happening? It is because of regular expressions. The way logs are being written it will never find a match with the original __bsd_syslog_verbose. Below script test both bsd_syslog_verbose settings. Originally we needed to have a ., but in reality we have a space in our logs, so we need to modify bsd_syslog_verbose.

#!/usr/bin/env python

import re

testline = 'May 13 06:24:36'

match = re.search('[^.]+\.[^.]+', testline)
if match:
    print 'Found:', match.group()
else:
    print 'Not found for bsd_syslog_verbose=[^.]+\.[^.]+'

match = re.search('[^.]+ [^.]+', testline)
if match:
    print 'Found:', match.group()
else:
    print 'Not found for bsd_syslog_verbose=[^.]+ [^.]+'

And we execute:

user@computer:$ python regex.py
Not found for bsd_syslog_verbose=[^.]+\.[^.]+
Found: May 13 06:24:36

More info here and some instructive regex google doc.

Pepephone 2G on Orange Pi 2G IOT

Recently I bought an Orange Pi 2G IOT. Pretty decent for $10 I would say, not a lot or RAM or CPU, but the interesting thing from my point of view is the 2G modem. Some pictures below with SIM card installed.


So the idea of this post is how to configure this device to connect to Pepephone network with a PIN on SIM. After some testing I finally came up with below wvdial.conf which works for me.

[Dialer defaults]
Modem = /dev/modem0
Baud = 30720000
Dial Command = ATDT
Init1 = ATE1
Init2 = AT+CPIN="XXXX"
Init3 = AT+CFUN=1
Init4 = AT+CGATT=1
Init5 = AT+CGDCONT=1,"IP","gprs.pepephone.com","",0,0
FlowControl = CRTSCTS
Phone = *99#
Username = " "
Password = " "
Stupid Mode = 1
Auto Reconnect = on

Pin entry above should be replaced with your SIM pin.
Execute running wvdial from cli and you should be able to see a new interface named ppp0. Below is a screenshot of both interfaces ppp0 and wlan0.More info here (in spanish).