Tag Archives: python

LeetCode problem 167

Back online after quite some time. Lately I’ve been playing with leetcode, so below is solution to problem 167. This problem is an easy one and no complex structures are needed in order to solve it.

#!/usr/bin/env python

''' LeetCode problem 167 
Given a 1-indexed array of integers numbers that is already sorted in non-decreasing order, 
find two numbers such that they add up to a specific target number. 
Let these two numbers be numbers[index1] and numbers[index2] where 1 <= first < second <= numbers.length.
2 <= numbers.length <= 3 * 104
-1000 <= numbers[i] <= 1000
numbers is sorted in non-decreasing order.
-1000 <= target <= 1000
The tests are generated such that there is exactly one solution. '''

def twoSum(numbers, target):
	for i in numbers:
		if (target - i) in numbers[numbers.index(i)+1:]:
			return [numbers.index(i), numbers.index(i) + numbers[numbers.index(i)+1:].index(target - i)+1]


#twoSum([-5, -3, -1, 0, 10], -1)
print(f"{twoSum([-5, -3, -1, 0, 10], -1)}")
#twoSum([2, 7, 11, 15], 9)
print(f"{twoSum([2,7,11,15], 9)}")
#twoSum([2, 3, 4], 6)
print(f"{twoSum([2, 3, 4], 6)}")
#twoSum([5, 25, 75], 100)
print(f"{twoSum([5, 25,75], 100)}")
#twoSum([0, 0, 3, 4], 0)
print(f"{twoSum([0, 0, 3, 4], 0)}")

I'll try to post more often from now on. I'll try to make it at least once a week.

Mapping Pirenaica with Folium

So back in September I cycled through the Pyrenees and decided to map it with Folium. Latitude and longitude info was retrieved from Strava gpx files and cleaned up using sed, grep and awk. Result is a file as below.

head Pirenaica_stage_1.csv
lat,lon
43.3057130,-1.9778780
43.3057130,-1.9778770
43.3057140,-1.9778770
43.3057230,-1.9778880
43.3060960,-1.9783310
43.3064990,-1.9788360
43.3065760,-1.9789890
43.3067630,-1.9791790
43.3069250,-1.9791730

Continue reading

Primitive way with Folium

So I discovered Folium about two months ago and decided to map the primitive way with it. Coordinates data is retrieved from Strava gpx files and cleaned up leaving only latitude and longitude as below.

head Camin_prim_stage1.csv
lat,lon
43.3111770,-5.6941620
43.3113360,-5.6943420
43.3114370,-5.6944600
43.3115000,-5.6945420
43.3116970,-5.6948090
43.3119110,-5.6950900
43.3122360,-5.6956830
43.3123220,-5.6958090
43.3126840,-5.6963740

Below is the python file we will use to retrieve data and create the map with the routes.

import folium
from pyspark.sql import SparkSession
from pyspark.sql.functions import col
spark = SparkSession.builder.master("local").getOrCreate()

# Change Spark loglevel
spark.sparkContext.setLogLevel('FATAL')

# Load the rides and ride_routes data from local instead of HDFS
position1 = spark.read.load("/home/user/Camin_prim_stage1.csv", format="csv", sep=",", inferSchema="true", header="true")
position2 = spark.read.load("/home/user/Camin_prim_stage2.csv", format="csv", sep=",", inferSchema="true", header="true")
position3 = spark.read.load("/home/user/Camin_prim_stage3.csv", format="csv", sep=",", inferSchema="true", header="true")

position = [position1, position2, position3]

m = folium.Map()
col=0
colArray=['red','blue','green']

# Check file was correctly loaded
for x in position:
# x.printSchema()
# x.show(2)

# Map position
coordinates = [[float(i.lat), float(i.lon)] for i in x.collect()]

# Make a Folium map
#m = folium.Map()
m.fit_bounds(coordinates, padding=(25, 25))
folium.PolyLine(locations=coordinates, weight=5, color=colArray[col]).add_to(m)
folium.Marker(coordinates[1], popup="Origin").add_to(m)
folium.Marker(coordinates[-1], popup="Destination").add_to(m)
col = col + 1
# Save to an html file
m.save('chamin_prim.html')

# Cleanup
spark.stop()

Continue reading

GPS on Orange Pi 2G IOT

Recently I got myself a GPS module to play with such as this. Not that I knew much about but felt like something nice to play with. First thing first is connecting the GPS module to the GPIO in Orange Pi 2G. In my case I decided to connect the GPS to ttyS2, asides from VCC (2.8V) and GND. This correspond to pins 1, 6, 8 and 10.
Vcc -> pin 1
GND -> pin 6
TxD -> pin 8
RxD -> pin 10

Pin 1 would be the one just below a small white arrow to the left of the SD card. Below two pictures of GPS connected to Orange Pi GPIO pins.


Another important thing is to cross TxD and RxD cables, that means connecting GPS TxD to Orange Pi RxD and GPS RxD to Orange Pi TxD.
Now that the GPS module is connected to the Orange Pi power it up and log into it. We will now check if the GPS mocule is working, in order to do this we will connect via minicom.

minicom -D /dev/ttyS2 -b 9600
Welcome to minicom 2.7

OPTIONS: I18n 
Compiled on Apr 26 2017, 00:45:18.
Port /dev/ttyS2, 19:10:00

Press CTRL-A Z for help on special keys

$GPRMC,191013.00,A,4252.31265,N,7833.00897,W,0.180,,040518,,,A*64
$GPVTG,,T,,M,0.180,N,0.333,K,A*29
$GPGGA,191013.00,4252.31265,N,7833.00897,W,1,06,2.16,296.4,M,51.3,M,,*44
$GPGSA,A,3,22,16,10,08,26,27,,,,,,,3.81,2.16,3.14*04
$GPGSV,3,1,10,01,19,244,,08,60,313,24,10,49,079,28,16,36,164,36*74
$GPGSV,3,2,10,18,42,250,22,20,07,038,,21,06,061,,22,09,195,30*7A
$GPGSV,3,3,10,26,09,158,30,27,69,072,27*70
$GPGLL,4252.31265,N,7833.7897,W,191013.00,A,A*7C

Continue reading

Fixing fail2ban

I had installed fail2ban but had noticed it wasn’t working blocking ssh brute force attacks. Attacks such as below.

grep sshd /var/log/auth.log | tail
Apr 29 08:06:17 sd-229337 sshd[20646]: pam_unix(sshd:auth): check pass; user unknown
Apr 29 08:06:17 sd-229337 sshd[20646]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=211-75-3-35.hinet-ip.hinet.net
Apr 29 08:06:18 sd-229337 sshd[20646]: Failed password for invalid user db2inst from 211.75.3.35 port 52724 ssh2
Apr 29 08:06:19 sd-229337 sshd[20646]: Connection closed by 211.75.3.35 [preauth]
Apr 29 08:18:21 sd-229337 sshd[20711]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=59-120-243-8.hinet-ip.hinet.net  user=root
Apr 29 08:18:25 sd-229337 sshd[20711]: Failed password for root from 59.120.243.8 port 34312 ssh2
Apr 29 08:18:25 sd-229337 sshd[20711]: Connection closed by 59.120.243.8 [preauth]
Apr 29 08:19:14 sd-229337 sshd[20713]: pam_unix(sshd:auth): authentication failure; logname= uid=0 euid=0 tty=ssh ruser= rhost=195-154-136-62.rev.poneytelecom.eu  user=root
Apr 29 08:19:16 sd-229337 sshd[20713]: Failed password for root from 195.154.136.62 port 24329 ssh2
Apr 29 08:19:16 sd-229337 sshd[20713]: Connection closed by 195.154.136.62 [preauth]

In order to fix this we need to modify /etc/fail2ban/filter.d/common.local and modify bsd_syslog_verbose entry. Substitute __bsd_syslog_verbose = (<[^.]+\.[^.]+>) for __bsd_syslog_verbose = (<[^.]+ [^.]+>).

grep bsd_syslog_verbose /etc/fail2ban/filter.d/common.local
#__bsd_syslog_verbose = (<[^.]+\.[^.]+>)
__bsd_syslog_verbose = (<[^.]+ [^.]+>)
__prefix_line = \s*%(__bsd_syslog_verbose)s?\s*(?:%(__hostname)s )?(?:%(__kernel_prefix)s )?(?:@vserver_\S+ )?%(__daemon_combs_re)s?\s%(__daemon_extra_re)s?\s*

Restart fail2ban and you shall now see IPs performing brute force attacks being blocked as below.

tail -30 /var/log/fail2ban.log | grep actions
2018-04-29 18:43:19,835 fail2ban.actions[28271]: WARNING [ssh] Unban 163.172.159.119
2018-04-29 18:43:20,742 fail2ban.actions[28519]: INFO    Set banTime = 1800
2018-04-29 18:43:20,936 fail2ban.actions[28519]: INFO    Set banTime = 600
2018-04-29 18:43:59,119 fail2ban.actions[28519]: WARNING [ssh] Ban 171.244.27.195
2018-04-29 18:46:05,286 fail2ban.actions[28519]: WARNING [ssh] Ban 5.188.10.185
2018-04-29 19:13:59,938 fail2ban.actions[28519]: WARNING [ssh] Unban 171.244.27.195
2018-04-29 19:14:50,026 fail2ban.actions[28519]: WARNING [ssh] Ban 171.244.27.195
2018-04-29 19:15:35,102 fail2ban.actions[28519]: WARNING [ssh] Ban 159.65.10.166
2018-04-29 19:16:06,167 fail2ban.actions[28519]: WARNING [ssh] Unban 5.188.10.185
2018-04-29 19:44:50,740 fail2ban.actions[28519]: WARNING [ssh] Unban 171.244.27.195
2018-04-29 19:45:35,821 fail2ban.actions[28519]: WARNING [ssh] Unban 159.65.10.166
2018-04-29 19:45:38,858 fail2ban.actions[28519]: WARNING [ssh] Ban 171.244.27.195

But why is this happening? It is because of regular expressions. The way logs are being written it will never find a match with the original __bsd_syslog_verbose. Below script test both bsd_syslog_verbose settings. Originally we needed to have a ., but in reality we have a space in our logs, so we need to modify bsd_syslog_verbose.

#!/usr/bin/env python

import re

testline = 'May 13 06:24:36'

match = re.search('[^.]+\.[^.]+', testline)
if match:
    print 'Found:', match.group()
else:
    print 'Not found for bsd_syslog_verbose=[^.]+\.[^.]+'

match = re.search('[^.]+ [^.]+', testline)
if match:
    print 'Found:', match.group()
else:
    print 'Not found for bsd_syslog_verbose=[^.]+ [^.]+'

And we execute:

 python regex.py 
Not found for bsd_syslog_verbose=[^.]+\.[^.]+
Found: May 13 06:24:36

More info here and some instructive regex google doc.