Author: Luis Perez

About Luis Perez

Website::Subscribe Feeds

Simple & Basic Home Firewall with Mikrotik

Posted on by Luis Perez

I will start this publication assuming you have a basic knowledge of the Mikrotik CLI and know your way around the interface.

This will also assume your firewall is empty, even though it may not be the case, you should be able to pick this one apart and chose what you like and skip what you don’t.

Some guides group the firewall rules by forward and input chains, but I like to group them by purpose, it seems better to me whenever you troubleshoot something.

Just know the Mikrotik firewall process rules in sequential order, this part is important.

So, to begin with the rules: 

/ip firewall filter
add action=accept chain=input comment="Accept established & related inputs" \
    connection-state=established,related
add action=accept chain=forward connection-state=established,related

The previous rules allows to avoid rechecking all packets passing through the router, this will save us CPU.

add action=drop chain=input comment="Drop invalid inputs & forwards" \
    connection-state=invalid
add action=drop chain=forward connection-state=invalid

The previous rules will reject any invalid or malformed packets.

add action=reject chain=input comment="Reject blacklisted" in-interface=\
    ether1-wan reject-with=icmp-network-unreachable src-address-list=\
    Blacklist

I like using blacklists to prevent malicious agents and IPs from reaching my router, this will make sense later.

add action=drop chain=forward comment="Drop all from WAN not DSTNATed" \
    connection-nat-state=!dstnat connection-state=new in-interface=ether1-wan

This rule will drop inbound forwards that are not ‘NATted’, the rule effectively blocks inbound traffic that hasn’t been explicitly mapped to a destination on your internal network through DNAT.

add action=accept chain=input comment="Accept inputs from the whitelist" \
    in-interface=ether1-wan src-address-list=Whitelist

You could skip this one, the purpose of this rule is to only allow the IPs you know to reach the router. Use this one carefully and please know there are better ways to access your router remotely, like through a VPN with wireguard. 

add action=add-src-to-address-list address-list="Unknown Admin" \
    address-list-timeout=1w chain=input comment="Log unknown admins" \
    dst-port=8291 in-interface=ether1-wan log=yes log-prefix="Unknown Admin" \
    protocol=tcp src-address=0.0.0.0/0
add action=accept chain=input comment="Accept unknown admins" dst-port=8291 \
    in-interface=ether1-wan protocol=tcp src-address=0.0.0.0/0

If you decide to keep the port 8291 open on the router, and I’m not saying you should, you may want to keep tabs on who is trying to access your router, this is the purpose of the rules above.

add action=accept chain=input comment="Accept inputs from home" in-interface=\
    bridge.home src-address=192.168.88.0/24
add action=accept chain=forward comment=\
    "Accept internet access for home devices" in-interface=home-bridge \
    out-interface=ether1-wan src-address=192.168.88.0/24

These rules allow your local LAN, configured on the bride called ‘bridge.home’ to reach your home router and to access the internet.

add action=add-src-to-address-list address-list=Blacklist \
    address-list-timeout=1w chain=input comment=\
    "Add forbidden attempts to the blacklist" dst-port=\
    21-23,25,53,80,110,135,139,443,445,587,1025,1352 in-interface=ether1-wan \
    protocol=tcp src-address=0.0.0.0/0 src-address-list=!Whitelist
add action=add-src-to-address-list address-list=Blacklist \
    address-list-timeout=1w chain=input dst-port=\
    1433,1433,1521,3306,3389,5060,5900,6001,8000-8080, in-interface=\
    ether1-wan protocol=tcp src-address=0.0.0.0/0 src-address-list=!Whitelist
add action=add-src-to-address-list address-list=Blacklist \
    address-list-timeout=1w chain=input dst-port=\
    53,69,161,135-139,445,593,1433-1434,1900 in-interface=ether1-wan \
    protocol=udp src-address=0.0.0.0/0 src-address-list=!Whitelist

This is the section of the blacklist, it will block any outside IP trying to connect to your router using any of the ports above.

If you are not exposing any services on your network to the outside world and there are IPs trying to access your network via any of the ports mentioned above, you’ll see them populate a blacklist in your router.

In my opinion, any public IP trying to reach your router on any of the ports above are looking for trouble.

add action=reject chain=input comment="Drop all from WAN" in-interface=\
    ether1-wan reject-with=icmp-network-unreachable
add action=reject chain=forward comment="Drop everything else" reject-with=\
    icmp-network-unreachable

The last two rules of the router are meant to drop any connection you may have not configured. It’s important not to skip these for security purposes.

Please note I’m not showing how to configure a NAT here, which you may need.

Replacing ZFS pools with boot partitions (In Proxmox)

Posted on by Luis Perez

I’ve created this document in order to have a clear guide to replace boot disks in a ZFS pool for proxmox, basically because the one on their documentation was not completely clear for me.

Source documentation: https://pve.proxmox.com/wiki/ZFS_on_Linux

In my case I was dealing with a Raid1 ZFS pool, both with bootable drives.

I wasn’t aware these drives had 3 partitions, which I had to replicate to the new drive in order to perform the proper replacement.

We replicate these partitions with the following commands:

sgdisk <healthy bootable device> -R <new device>    -  (use /dev/diskname)

sgdisk -G <new device>  - (use /dev/diskname)

The the second command will make sure the new partitions that have been copied from the remaining surviving drive, have unique GUIDs, it’s a bad idea to have disks with cloned GUIDs.

In the example above we see that Nvme0n1 is the remaining disk in the array, which is in good state.

Nvme0n2 is the new one, the one we are going to used to replace the failed one.

Knowing this we run the following commands:

sgdisk /dev/nvme0n1 -R /dev/nvme0n2

sgdisk -G /dev/nvme0n2

The last command should output: The operation has completed successfully.

After, we should see the partitions replicated:

Now, we need to add the partition 3 to the ZFS pool array. Previously I made the mistake of adding the complete disk, which would destroy the partitions created and will not let you install the boot partition into the disk.

Avoid that mistake, what we need to add is the disk to the array, not the complete disk.

Lets move now to identify the partition ID we want to replicate:

ls -lh /dev/disk/by-id/

We know the new disk is the nvme0n2, and we know the partition is nvme0n2p3, so the ID we’ll use now its:

nvme-VMware_Virtual_NVMe_Disk_VMware_NVME_0000_2-part3

The command we need to now follow its:

# zpool replace -f <pool> <failed disk id> <new zfs partition>

From the first image in the document, we know the failed partition has this ID: 15896803577790237437

The resulting command should be:

zpool replace -f rpool 15896803577790237437 nvme-VMware_Virtual_NVMe_Disk_VMware_NVME_0000_2-part3

Do NOT do it like this:

zpool replace -f rpool 15896803577790237437 /dev/nvme0n2p3

The zpool replace command will start a resilvering process, which you should monitor until its 100% complete before moving forward.

We can monitor this process with the command:

watch zpool status -v

Once this process is completed, you can move on to install the boot files in the p2 partition for the drive

First we need to validate if we are using UEFI or GRUB with the following command:

proxmox-boot-tool status

You need to validate if the system says you are booting with legacy bios or UEFI.

# proxmox-boot-tool format <new disk's ESP>

In the example we are using, we know the boot partition should be /dev/nvme0n2p2, so following the example above the next command should be:

proxmox-boot-tool format /dev/nvme0n2p2

# proxmox-boot-tool init <new disk's ESP>  [grub] (optional)

After formatting the partition, we proceed to install the boot files, if we are using grub the command should be:

proxmox-boot-tool init /dev/nvme0n2p2 [grub]

If we are using UEFI, the command should be:

proxmox-boot-tool init /dev/nvme0n2p2

Then we proceed to clean the previous boot entries that are no longer relevant with the following command:

proxmox-boot-tool clean

You can now proceed to validate if the boot partitions have been correctly installed with the following commands:

proxmox-boot-tool status

cat /etc/kernel/proxmox-boot-uuids

The output should look similar to this (Legacy Bios example)

Since this is a raid1 pool with 2 disk, we should only see two lines per output.

You should now be able to boot from both drives!

How to create VLANs with Mikrotik – Part 1

Posted on by Luis Perez

I’ve found when you begin to work with Mikrotik, creating and handling VLANs can get a bit daunting (Especially when you come from a Cisco background) so I’ve decided to make a 3 part tutorial on how to create VLANs in Mikrotik:

Part 1: The easy and clunky way.

Part 2: The right way without using the chip switch

Part 3: The right way to use the chip switch.

This part is for those who need to use VLANs in a home environment and just want to get it done and out of the way. I’ve called it the clunky and easy way because even though it works, It’s not the proper way of handling and creating VLANs in Mikrotik routers, so this is mostly for beginners and home users. But it works. It’s clunky because you’ll need to create a bridge for every VLAN you have.

I’ve found no downsides while using this method in a home environment or a home lab, I haven’t tried this method in production environments and I strongly advise against it, mostly because it can get overly complicated as it grows and you could overutilize the CPU of your router when you could be using the switch chip.

In this scenario, I’ll be configuring over GNS3 a Mikrotik router with 3 VLANs, and a trunk port, and I’ll be connecting it to a Cisco Switch just to prove that it works.

Each device will have devices connected to it working with different VLANs.

I’ll be also sharing the config files for the Mikrotik and the cisco switch. Please take into consideration that there is no firewall nor security configured anywhere in this lab, so don’t try to apply this method blindly without setting up a firewall or password protecting your devices.

I’ll be also assuming you know how to read the config files on your Mikrotik devices and Cisco Switch and you know how the configuration on Mikrotik relates to the GUI.

Now for starters on the Mikrotik, let us begin with the Trunk Port.

Some people like to create the VLANs directly on a physical interface, I rather create the VLANs over a Bridge which I usually call the “Trunk Bridge” this is because I may want to have more than one trunk port.

I will begin with a clean slate configuration that only has a DHCP client configured the ether1.

To begin, we’ll create Bridge and then attach it to the interface we want to work with. I’ll be creating a bridge called “bridge-trunk” that will be attached to the ethernet interface 5 of the Mikrotik:

For this we’ll type on the Mikrotik’s terminal:

/interface bridge
add name=bridge-trunk

/interface bridge port
add bridge=bridge-trunk interface=ether5

And we’ll see on Winbox something like this:

Now that we have our trunk bridge, we can create our VLANs and attach them to our trunk bridge.

/interface vlan
add interface=bridge-trunk name="vlan 2" vlan-id=2
add interface=bridge-trunk name="vlan 3" vlan-id=3
add interface=bridge-trunk name="vlan 4" vlan-id=4

And we’ll get something like this:

Now, we’ll create a bridge for every VLAN we are going to use and we’ll connect it to each VLAN we’ve created, and then, we’ll connect them to our desired ports.

/interface bridge
add name=br-vlan2
add name=br-vlan3
add name=br-vlan4
/interface bridge port
add bridge=br-vlan2 interface="vlan 2"
add bridge=br-vlan3 interface="vlan 3"
add bridge=br-vlan4 interface="vlan 4"
add bridge=br-vlan4 interface=ether4
add bridge=br-vlan3 interface=ether3
add bridge=br-vlan2 interface=ether2

From here, we already have our access ports and trunk port working, however, we still need to configure IPs and the DHCP server in order for everything to work properly.

In case you’re following this in a physical or virtual lab, I’ll leave the rest of the commands and the configuration file for both the Mikrotik and Cisco Switch.

In order for our connected devices to work, we need to set up the IP of the bridge interfaces for each VLAN, a DHCP Server, and our NAT.

/ip address
add address=10.0.2.1/24 interface=br-vlan2 network=10.0.2.0
add address=10.0.3.1/24 interface=br-vlan3 network=10.0.3.0
add address=10.0.4.1/24 interface=br-vlan4 network=10.0.4.0
add address=10.100.0.1/24 interface=bridge-trunk network=10.100.0.0

/ip pool
add name=dhcp_pool0 ranges=10.0.2.2-10.0.2.254
add name=dhcp_pool1 ranges=10.0.3.2-10.0.3.254
add name=dhcp_pool2 ranges=10.0.4.2-10.0.4.254
/ip dhcp-server
add address-pool=dhcp_pool0 disabled=no interface=br-vlan2 name=dhcp1
add address-pool=dhcp_pool1 disabled=no interface=br-vlan3 name=dhcp2
add address-pool=dhcp_pool2 disabled=no interface=br-vlan4 name=dhcp3

/ip dhcp-client
add disabled=no interface=ether1-wan
/ip dhcp-server network
add address=10.0.2.0/24 dns-server=10.0.2.1 gateway=10.0.2.1
add address=10.0.3.0/24 dns-server=10.0.3.1 gateway=10.0.3.1
add address=10.0.4.0/24 dns-server=10.0.4.1 gateway=10.0.4.1
/ip dns
set allow-remote-requests=yes
/ip firewall nat
add action=masquerade chain=srcnat

Now we can see PC 1 and PC 8 which are on different VLANs and on each end of the setup being able to talk to each other and to get Internet connectivity

And if you would like to study each configuration, here is the ending configuration file for the Mikrotik and the Cisco Switch.

cisco-switchDownload
mikrotik-easy-clunkyDownload

If you are interested in the GNS3 lab, send me an email and I’ll gladly share it with you.

I hope this has been informative for you, happy trails!

How to control the lights of your Xiaomi Gateway 3 with OpenLumi.

Posted on by Luis Perez

This is a guide on how to set up a Xiaomi Gateway 3 flashed with OpenLumi for you to control its LED lights through Home Assistant.

First things first, props to the people who allowed me to do it, I’ve originally followed these two guides written in Russian to do it myself:

If you get stuck, you may want to check them out for reference.

Before moving forward, you need to have configured MQTT on your Home Assistant, if you haven’t done that, please follow one of these guides to set it up first:

Also, I’m assuming that you already have some experience with Home Assistance and know how to work on the Linux terminal, I’m also assuming you’ve already flashed your Xiaomi Gateway with OpenLumi. If you haven’t, please look at my previous posts.

Now, you need to have installed some packages first, so open the terminal of your Gateway and install these packages:

opkg update && opkg install node git-http mpg123 mpc mpd-full

Now you need to install Lumi:

mkdir /opt
cd /opt
git clone https://github.com/Beetle-II/lumi.git
cd lumi
cp config_example.json config.json

Now we’ll use “vi” to edit the config.json file:

{
  "sensor_debounce_period": 300,
  "sensor_treshhold": 50,
  "button_click_duration": 300,
          
  "homeassistant": true,
  "tts_cache": true,
  "sound_channel": "Master",
  "sound_volume": 50,
  "mqtt_url": "mqtt://[HA IP ADDRESS HERE]",
  "mqtt_topic": "lumi", #Use the name of your gateway
  "use_mac_in_mqtt_topic": true, 
  "mqtt_options": {
    "port": 1883,
    "username": "login here",
    "password": "password here",
    "keepalive": 60,
    "reconnectPeriod": 1000,
    "clean": true,
    "encoding": "utf8",
    "will": {
      "topic": "lumi/state",
      "payload": "offline",
      "qos": 1,
      "retain": true
    }
  }
}

I highly advise you to use the “true” value on the “use_mac_in_mqtt_topic”, if by any chance you have more than one gateway, this will help you differentiate all of them on your Home Assistant.

Also, make sure to change all “lumi” values with the desired name of your Gateway, it will help you differentiate the devices on your home assistance in case you have more than one.

After configuring the config.json file, we will need to start our service, so we launch on the terminal:

node /opt/lumi/lumi.js

That might result in an error, so after getting feedback from your gateway, use control+c or control+z to exit.

After that, we type on the terminal:

cd /opt/lumi
chmod +x lumi
cp lumi /etc/init.d/lumi
/etc/init.d/lumi enable
/etc/init.d/lumi start

Then type again:

node /opt/lumi/lumi.js

If you see something like this:

It means you have set it up correctly and it’s now done, you should be able to see your device on your Home Assistant Integrations section, here is how mine looks:

In my case, I have two gateways configured, if you have only one you should see half as much.

That’s it! Happy setups!

Using Node-Red to push media and TTS messages to your Xiaomi Gateway 3

Posted on by Luis Perez

Before starting, this guide assumes you’ve been able to do the following things on your Gateway:

  • You’ve been able to flash your Xiaomi Gateway 3 with OpenLumi
  • You’ve been able to connect to your Gateway via SSH and it has a static IP
  • You’ve been able to set up an MPD Server on your OpenLumi and know it’s working because you’ve tested it with Auremo.
  • You’ve been able to upload the mp3 files you need and set up your MPD device on Home Assistant using Yaml.
  • You have Node-Red configured and have some degree of experience with it.

If you have not been able to do these things, please check out my previous post.

What was particularly hard to set up while working with the MPD Daemon on my Lumi Gateway was being able to figure out how to send Text To Speech (TTS) commands and play audio files from Node-Red.

Before moving forward, make sure you can play the files you have uploaded to your gateway using Auremo and you can send TTS commands directly from Home Assistant.

If you can do these things and you are also able to work with Node-Red within your home assistant, then please, continue.

If you need help setting up Node-Red Home Assistant, I highly recommend the Node-Red videos from The Hookup Youtube Channel.

Now, to set up our gateway on Node-Red we’ll need to use the following nodes: Trigger, Inject and call service.

We will use inject to test commands and trigger our Node-Red flows.

The first thing we need to do is to learn to manage the volume of what we want to hear in your Gateway. To do this I use the following, double click on your call service node and fill it up will the following information:

Name: Volume (Or whatever you feel comfortable with)
Server: Home Assistant
Domain: media_player
Service: volume_set
Entity: Xiaomi Gateway or use the name you chose for your gateway
On Data, make sure you chose JSON and type the volume you desire, this one goes from 0.0 to 1, on this case I've chosen 0.6, {"volume_level":"0.6"}

After you can click now on the red button labeled “Done” in the upper right corner of the node window.

You should have something like this:

After this step we would like our gateway to be able to say or play something, I will choose now to play a sound, so, again will I use the call service node for this.

Then we connect the nodes:

Then I added a new call service node and I’ve called it “Ding”, so here is the configuration as follows:

Name: Ding (Or whatever you feel comfortable with)
Server: Home Assistant
Domain: media_player
Service: play_media (Dont confuse this with media_play)
Entity: Xiaomi Gateway or use the name you chose for your gateway
On Data, make sure you chose JSON and type the name of an uploaded mp3 on your /mpd/music folder, on my case I'm choosing {"media_content_type":"music","media_content_id":"Doorbell.mp3"}

After this step, we press the red button “Done” and then “Deploy”, you’ll find the deploy button on the top right corner of Node-Red.

You should be able to play the sound “Doorbell.mp3” or whatever file you’ve chosen using the inject node.

Now to make it say something, you need to have something like this, again we’ll use the call service node, but first, notice I’ve also placed a trigger node between the played audio node “Ding” and the TTS node “Someone at the door”. This is because if you don’t use the trigger, Node-Red will not wait until the sounds finish playing before playing the TTS message. You’ll only hear the “Someone at the door” node without the “Ding” node.

Now, to make it say something, make sure you have something like this on your “Someone at the door” call service node:

Notice we are using J: expression instead of {} JSON this time. The node configuration is as follows:

Name: Someone at the door (Or whatever you feel comfortable with)
Server: Home Assistant
Domain: tts
Service: cloud_say or google_say (Try the one you like the most)
Entity: Xiaomi Gateway or use the name you've chosen for your gateway
On Data, make sure you select J:Expression  and type {"message":"There is someone at the door"}.

I believe it also works if you chose JSON.

After you’ve configured the node, hit the red done button and the deploy button and your flow should be ready, you should be able to push TTS and sound messages from Node-Red to your Xiaomi Gateway now.

Configuring Media Player Daemon on the Xiaomi Gateway 3

Posted on by Luis Perez
DGNWG05LM / ZHWG11LM

Before beginning, It’s worth noting I’ve already flashed OpenWRT on my Xiaomi Gateway 3 (DGNWG05LM / ZHWG11LM)

I’m assuming you’ve flashed already OpenLumi on your gateway, if you haven’t, please follow the guides below to do it:

This is how you root it:
https://openlumi.github.io/gain_root.html
And this is how you flash OpenWRT on it:
https://openlumi.github.io/

Once your device is rooted, flashed with OpenWRT and you have ssh access to it, you’ll find your Xiaomi Gateway has become pretty useless unless you have an actual plan for it. (As it was my case)

In my case, I wanted to be able to push media files and TTS messages to the gateway using Home Assistant and Node-Red. I’ll try to explain the Node-Red part in a separate post, but for the time being, I’ll focus on the MPD Server setup.

A great part of what I’m doing is based on a guide I’ve found in Russian, but I’ve found it hard to understand so I thought it would be a good idea to write this one in another language.

After you’ve managed to gain root access, you need to begin by installing some packages on your gateway, to do this we will run on the terminal :

opkg update && opkg install node git-http mpg123 mpc mpd-full

Once you do, we need to create the files and directories for MPD to work, so go ahead and type in your gateway’s terminal: 

cd /
mkdir mpd
cd mpd 
mkdir music
mkdir playlists
touch database
touch log
touch /etc/mpd.state

Once you’ve done this, we will create the configuration file for MPD, to do this we will make the mpd.conf file on the /etc/

I’ve used “vi” for this, if you don’t know how to use vi to edit files, please check this out: https://www.redhat.com/sysadmin/introduction-vi-editor

This is how I did it:

I first created a backup of the original file and deleted the original one:

cp /etc/mpd.conf /etc/mpd.conf.bkp
rm /etc/mpd.conf

Then, I created my new mpd.conf file with the command:

vi /etc/mpd.conf

Once you’re ready to edit, you can go ahead and paste the following config info:


state_file "/mpd/state" # Read the note at the bottom "How do I set the MPD to keep the volume level?"
music_directory "/mpd/music" # Specify the path to the music folder, in my case the mpd folder is in the root
playlist_directory "/mpd/playlists" # Specify the path to the playlists folder, in my case the mpd folder is in the root
db_file "/mpd/database" # Specify the path to the database database, in my case the mpd folder is at the root
#log_file "/mpd/log" # Uncomment the line if you want to write to the log
#log_level "default" # Uncomment the line if you want to write to the log
bind_to_address "any"
port "6600"
user "root"
group "root"
auto_update "yes"
auto_update_depth "3"
filesystem_charset "UTF-8"
id3v1_encoding "UTF-8"
audio_output {
  type "alsa"
  name "My ALSA Device"
  device "hw:0,0"
  mixer_type "software"
  mixer_control "Master"
  mixer_device "default"
}

After saving it (Remember, to save on vi you need to type ESC and type :wq!, you can check the configuration by typing:

cat /etc/mpd.conf

And you should have something similar to this:

Then to apply the changes run:

/etc/init.d/mpd stop
/etc/init.d/mpd start

After this step, I’ve uploaded an mp3 file to the /mpd/music/ folder to test the functionality, for obvious reasons I can’t share that file, but we’ll call it doorbell.mp3, there are plenty of sites that offer these kinds of files for free.

Now to test if the MPD server is working you can download Auremo from this website:

https://code.google.com/archive/p/auremo/downloads

After installing Auremo, make sure you set it up using the IP of your MPD server (Your Gateway’s IP) to test its functionality, if you can see your mp3 file and you’re able to play it from your computer, you should be able to make it work on Home Assistant as well.

After that, you should be ready to use your MPD daemon on your home assistant as long as you configure the gateway on your configuration.yaml file, my configuration is similar to this:

media_player:
  - platform: mpd
    name: Xiaomi Lumi Gateway (Use whatever name makes you happy)
    host: 192.168.0.100 (Make sure you use the IP assigned to your gateway and you have a static ip)
    port: 6600
    scan_interval: 30

After this, restart Home Assistant.

You should now be able to push TTS commands on your Home Assistant by going to:

Settings >> Home Assistant Cloud >> Text to Speech and clicking on “Try”

From there, select your newly added Xiaomi Lumi Gateway device and try to make it say a few things. 🙂

Right now, just like this, it’s ready for me to push commands, in my case I’ll use it with Node-Red to push alerts and voice commands.

I’ll be showing how I did this in a later post.