ntzyz's space

Early Experience with ZFS RAIDZ Expansion

Sun, 17 Nov 2024 00:00:00 +0000

Compiling and Installing OpenZFS 2.3.0

By now OpenZFS 2.3.0 has not been officially released yet, with the latest version being 2.3.0-rc3. Therefore, you need to compile it yourself. The process is relatively simple:

First, install a relatively new kernel (the Debian kernel version on my NAS is 6.1.0):

`1`	`apt install linux-image-6.9.7+bpo-amd64 linux-headers-6.9.7+bpo-amd64`

Once completed, follow the relevant steps in the official ZFS documentation to compile it.

apt install alien autoconf automake build-essential debhelper-compat dh-autoreconf dh-dkms dh-python dkms fakeroot gawk git libaio-dev libattr1-dev libblkid-dev libcurl4-openssl-dev libelf-dev libffi-dev libpam0g-dev libssl-dev libtirpc-dev libtool libudev-dev parallel po-debconf python3 python3-all-dev python3-cffi python3-dev python3-packaging python3-setuptools python3-sphinx uuid-dev zliblg-dev
git clone https://github.com/openzfs/zfs
cd ./zfs
git checkout zfs-2.3.0-rc3
sh autogen.sh
./configure
make -s -j$(nproc) native-deb-utils
cd ..
rm openzfs-zfs-dracut_*.deb openzfs-zfs-initramfs_2.3.0-1_all.deb
sudo dpkg -i ./openzfs-zfs-zed_2.3.0-1_amd64.deb ./openzfs-zfs-dkms_2.3.0-1_all.deb ./openzfs-libuutil3_2.3.0-1_amd64.deb ./openzfs-libzfs6_2.3.0-1_amd64.deb ./openzfs-libnvpair3_2.3.0-1_amd64.deb ./openzfs-zfsutils_2.3.0-1_amd64.deb

Once the above steps are successfully completed, simply restart the system. After restarting, you may need to manually run zpool import -a to re-import the pools.

Upgrading a Zpool to Use RAIDZ Expansion

This step is very straightforward. Assuming the zpool you need to upgrade is named sata-critical, you can do it with a single command:

`1`	`zpool upgrade sata-critical`

Adding Disks to a Zpool

This step is also very simple:

`1`	`zpool attach sata-critical raidz2-0 /dev/disk/by-id/ata-HGST_HUS726XXXXXXXXX_XXXXXXXX`

After that, you can use zpool status to check the progress.

$ zpool status sata-critical
  pool: sata-critical
 state: ONLINE
  scan: scrub repaired 0B in 05:19:56 with 0 errors on Sun Nov 17 10:31:03 2024
expand: expansion of raidz2-0 in progress since Sun Nov 17 18:48:14 2024
	1.48T / 17.9T copied at 171M/s, 8.30% done, 1 days 03:56:00 to go
config:

	NAME                                   STATE    READ WRITE CKSUM
	sata-critical                         ONLINE       0     0     0
	 raidz2-0                             ONLINE       0     0     0
	   ata-HGST_HUS726XXXXXXXXX_XXXXXXXX  ONLINE       0     0     0
	   ata-HGST_HUS726XXXXXXXXX_XXXXXXXX  ONLINE       0     0     0
	   ata-HGST_HUS726XXXXXXXXX_XXXXXXXX  ONLINE       0     0     0
	   ata-HGST_HUS726XXXXXXXXX_XXXXXXXX  ONLINE       0     0     0
	   ata-HGST_HUS726XXXXXXXXX_XXXXXXXX  ONLINE       0     0     0

errors: No known data errors

Some Minor Drawbacks of Expansion

Suppose you set up a RAIDZ2 zpool using five new 6TB disks. Theoretically, the available space should be 15.643519TB (calculation reference), but after completion, the actual available space is only 13.08TB. The specific results are as follows:

$ zfs list sata-critical 
NAME            USED  AVAIL  REFER  MOUNTPOINT
sata-critical  8.66T  4.42T  8.66T  /sata-critical
$ zpool list sata-critical -v
NAME                                    SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP    HEALTH  ALTROOT
sata-critical                          27.3T  17.9T  9.39T        -         -     2%    65%  1.00x    ONLINE  -
  raidz2-0                             27.3T  17.9T  9.39T        -         -     2%  65.6%      -    ONLINE
    ata-HGST_HUS726XXXXXXXXX_XXXXXXXX  5.46T      -      -        -         -      -      -      -    ONLINE
    ata-HGST_HUS726XXXXXXXXX_XXXXXXXX  5.46T      -      -        -         -      -      -      -    ONLINE
    ata-HGST_HUS726XXXXXXXXX_XXXXXXXX  5.46T      -      -        -         -      -      -      -    ONLINE
    ata-HGST_HUS726XXXXXXXXX_XXXXXXXX  5.46T      -      -        -         -      -      -      -    ONLINE
    ata-HGST_HUS726XXXXXXXXX_XXXXXXXX  5.46T      -      -        -         -      -      -      -    ONLINE

After a bit of research, the reason seems to be related to the implementation of RAIDZ expansion. For more details, you can refer to this article: ZFS RAIDZ Expansion Is Awesome but Has a Small Caveat

ZFS Expansion: turn a mirror in to a RAID

Mon, 26 Aug 2024 00:00:00 +0000

In a previous article, I discussed using MergerFS to expand storage. However, as performance demands increased (and with too much empty space in the new HomeLab and increasingly cheap second-hand enterprise CMR disks), I decided to switch to a more mainstream RAID configuration to achieve both disaster recovery and expansion. After all, union filesystems like MergerFS are essentially similar to RAID0, making it difficult to guard against sudden failures.

At the beginning, I planned to categorize my home storage into two levels of usability: the first being high-reliability storage for critical data, and the second being less reliable storage with higher tolerance for data loss. Based on this approach, I configured my six HDDs as follows:

High-Reliability Storage: RAIDz2, consisting of four brand new WD DC HC310 6TB SATA HDDs from JD.com, providing a total of 12TB storage capacity.
Low-Reliability Storage: ZFS mirror, consisting of two second-hand WD DC HC530 6TB SAS HDDs from Xianyu (platform in China like e-bay), providing a total of 12TB storage capacity.

After some time, I became a PT player and started downloading more and more data. This increased my demand for low-reliability storage, leading me to purchase two more 12TB disks to expand this pool.

The entire process can actually be quite simple: take out four disks to create a new pool, and then use zfs send | zfs receive to copy the contents over. However, I didn’t have four spare disks to create a new pool, so I had to come up with some tricky workarounds. RAIDz2 requires at least four disks, with two disks storing parity and the remaining N-2 disks storing data. This mode actually allows for operation with just two disks online. Based on this idea, I used a combination of real disks and dummy files to fool the system into creating the pool. Then, by offlining the dummy files, I could get a degraded pool. The complete process was as follows:

Create two 12TB files in /tmp using truncate, and set up a RAIDz2 with the two new 12TB hard disks and these files.
Offline the two files in /tmp, causing the new pool to enter a degraded state but remain operational.
Use zfs send to transfer the data from the old pool to the new pool.
Remove one disk from the old pool and replace one of the dummy files. Wait for the resliver to complete.
Remove the other disk from the old pool and replace the other dummy file. Wait for the resliver to complete.

This process took approximately four days, with minimal downtime. The NAS was rebooted a few times during the disk replacements, but the overall result was excellent.

~~Of course, there are inherent risks. If any disk failure occurs during this process, there is a chance that all the data could be lost (and end up “in the cloud” lol).~~

Run Ray-Tracing in Browser with WebAssembly

Sun, 07 Apr 2024 00:00:00 +0000

Recently, within the small team at my company, I conducted a sharing session on WebAssembly, a technology that isn’t exactly trending but also isn’t very popular. I roughly discussed the development processes of asm.js and WebAssembly and, using the code from the famous “Ray Tracing in One Weekend”, I demonstrated the relatively superior performance of WebAssembly (superior, of course, compared to JavaScript, which is rather unimpressive). I myself had previously played around with a small experiment running CoreMark on WebAssembly, which can brute-force a CPU performance benchmark, but I haven’t recorded the details of the process that I went through. So, I’m considering writing a document to record it all ~~, just in case I have to start over from scratch in the future~~.

Installing Emscripten

Emscripten is a toolchain for WebAssembly, which we need to install manually. The process isn’t complicated; you can check the official documentation for detailed installation instructions. In short, we just need to execute the following commands to complete the installation and activation:

git clone https://github.com/emscripten-core/emsdk.git
cd emsdk
./emsdk install latest
./emsdk activate latest
source ./emsdk_env.sh

It’s clear that this installation method will only work in the current shell session. If needed, you can add source ./emsdk_env.sh to the shell’s initialization script, which I won’t elaborate on here.

Getting and Modifying the Source Code of “Ray Tracing in One Weekend”

This is a fairly well-known introductory book on ray tracing, which can be read via this link: Ray Tracing in One Weekend. You can also find the final implemented code in the source repository of the book. For our purposes, I will use the provided sample answer directly rather than starting from scratch.

Compiling the original code directly into WebAssembly would certainly work, but I’m planning to do some optimization:

The final implemented code’s method of outputting images is to print PPM to std::cout. We can change this to allocate a bitmap directly in WASM’s memory and update it;
The final implemented code’s progress output method is to use std::clog to output Scanlines remaining: xx. We can change this to calling a JavaScript function that outputs progress to the JavaScript console and draws the aforementioned bitmap onto a canvas, allowing everyone to directly see the progress.

First, let’s deal with camera.h, where I added a definition for extern void report_number(number);. The specific implementation will be supplemented in JavaScript later to achieve the second point of optimization mentioned above.

@@ -17,16 +17,18 @@
 #include "hittable.h"
 #include "material.h"
 
-#include <iostream>
-
+extern void report_number(int);
 
 class camera {
   public:
     double aspect_ratio      = 1.0;  // Ratio of image width over height
     int    image_width       = 100;  // Rendered image width in pixel count
+    int    image_height;             // Rendered image height
     int    samples_per_pixel = 10;   // Count of random samples for each pixel
     int    max_depth         = 10;   // Maximum number of ray bounces into scene
 
+    uint8_t *image_buffer = nullptr; // Bitmap for rendered image.
+
     double vfov     = 90;              // Vertical view angle (field of view)
     point3 lookfrom = point3(0,0,-1);  // Point camera is looking from
     point3 lookat   = point3(0,0,0);   // Point camera is looking at
@@ -37,26 +39,25 @@ class camera {
 
     void render(const hittable& world) {
         initialize();
-
-        std::cout << "P3\n" << image_width << ' ' << image_height << "\n255\n";
+        auto p = this->image_buffer;
 
         for (int j = 0; j < image_height; ++j) {
-            std::clog << "\rScanlines remaining: " << (image_height - j) << ' ' << std::flush;
+            if (j % 10 == 0)
+              report_number(image_height - j);
+            // std::clog << "\rScanlines remaining: " << (image_height - j) << ' ' << std::flush;
             for (int i = 0; i < image_width; ++i) {
                 color pixel_color(0,0,0);
                 for (int sample = 0; syyample < samples_per_pixel; ++sample) {
                     ray r = get_ray(i, j);
                     pixel_color += ray_color(r, max_depth, world);
                 }
-                write_color(std::cout, pixel_color, samples_per_pixel);
+                write_color(pixel_color, samples_per_pixel, p);
+                p += 4;
             }
         }
-
-        std::clog << "\rDone.                 \n";
     }
 
   private:
-    int    image_height;    // Rendered image height
     point3 center;          // Camera center
     point3 pixel00_loc;     // Location of pixel 0, 0
     vec3   pixel_delta_u;   // Offset to pixel to the right
@@ -68,6 +69,7 @@ class camera {
     void initialize() {
         image_height = static_cast<int>(image_width / aspect_ratio);
         image_height = (image_height < 1) ? 1 : image_height;
+        this->image_buffer = new uint8_t[image_height * image_width * 4];
 
         center = lookfrom;

And for color.h, which is responsible for handling colors:

@@ -13,8 +13,6 @@
 
 #include "vec3.h"
 
-#include <iostream>
-
 using color = vec3;
 
 inline double linear_to_gamma(double linear_component)
@@ -22,7 +20,7 @@ inline double linear_to_gamma(double linear_component)
     return sqrt(linear_component);
 }
 
-void write_color(std::ostream &out, color pixel_color, int samples_per_pixel) {
+void write_color(color pixel_color, int samples_per_pixel, uint8_t *buffer) {
     auto r = pixel_color.x();
     auto g = pixel_color.y();
     auto b = pixel_color.z();
@@ -40,9 +38,11 @@ void write_color(std::ostream &out, color pixel_color, int samples_per_pixel) {
 
     // Write the translated [0,255] value of each color component.
     static const interval intensity(0.000, 0.999);
-    out << static_cast<int>(256 * intensity.clamp(r)) << ' '
-        << static_cast<int>(256 * intensity.clamp(g)) << ' '
-        << static_cast<int>(256 * intensity.clamp(b)) << '\n';
+
+    *(buffer + 0) = static_cast<int>(256 * intensity.clamp(r));
+    *(buffer + 1) = static_cast<int>(256 * intensity.clamp(g));
+    *(buffer + 2) = static_cast<int>(256 * intensity.clamp(b));
+    *(buffer + 3) = 255;
 }

Finally, we will make some modifications to the entry file main.cc, adding a few functions for JavaScript to obtain some parameters of the canvas, and we also renamed the main function:

@@ -9,6 +9,7 @@
 // along with this software. If not, see <http://creativecommons.org/publicdomain/zero/1.0/>.
 //==============================================================================================
 
+#include <emscripten.h>
 #include "rtweekend.h"
 
 #include "camera.h"
@@ -17,8 +18,24 @@
 #include "material.h"
 #include "sphere.h"
 
+camera& get_camera() {
+  static camera cam;
+  return cam;
+}
+
+EMSCRIPTEN_KEEPALIVE int get_height() {
+  return get_camera().image_height;
+}
+
+EMSCRIPTEN_KEEPALIVE int get_width() {
+  return get_camera().image_width;
+}
+
+EMSCRIPTEN_KEEPALIVE uint8_t* get_buffer() {
+  return get_camera().image_buffer;
+}
 
-int main() {
+EMSCRIPTEN_KEEPALIVE int run() {
     hittable_list world;
 
     auto ground_material = make_shared<lambertian>(color(0.5, 0.5, 0.5));
@@ -61,7 +78,7 @@ int main() {
     auto material3 = make_shared<metal>(color(0.7, 0.6, 0.5), 0.0);
     world.add(make_shared<sphere>(point3(4, 1, 0), 1.0, material3));
 
-    camera cam;
+    camera& cam = get_camera();
 
     cam.aspect_ratio      = 16.0 / 9.0;
     cam.image_width       = 1200;
@@ -77,4 +94,5 @@ int main() {
     cam.focus_dist    = 10.0;
 
     cam.render(world);
+    return 0;
 }

After completing these changes, we should be able to compile the code into a wasm binary!

1
2
3

$ emcc -o hello.wasm main.cc -Wall --no-entry -s ERROR_ON_UNDEFINED_SYMBOLS=0 -O3
$ file main.wasm 
main.wasm: WebAssembly (wasm) binary module version 0x1 (MVP)

With that, all the work related to the “native” side has been completed.

Writing JavaScript Glue Code

As is well-known, WebAssembly code cannot be directly executed by browsers, hence we need to prepare some “glue” code to load the wasm file and provide the necessary external definition functions:

WebAssembly.instantiateStreaming(fetch("main.wasm"), {
  env: {
    _Z13report_numberi: function (num) {
      console.log(`report_number called: ${num}`);
      const resultAddress = results.instance.exports._Z10get_bufferv();
      const width = results.instance.exports._Z9get_widthv();
      const height = results.instance.exports._Z10get_heightv();

      const memoryView = new Uint8ClampedArray(results.instance.exports.memory.buffer, resultAddress, width * height * 4)
      postMessage({
        kind: 'IMAGE_DATA',
        width,
        height,
        memoryView,
      })
    }
  },
  wasi_snapshot_preview1: {
    clock_res_get: function () { console.info('warning: clock_res_get called, function not implemented'); return 0 },
    clock_time_get: function () { console.info('warning: clock_time_get called, function not implemented'); return 0 },
    fd_write: function () { console.info('warning: fd_write called, function not implemented'); return 0 },
    fd_read: function () { console.info('warning: fd_read called, function not implemented'); return 0 },
    fd_close: function () { console.info('warning: fd_close called, function not implemented'); return 0 },
    fd_seek: function () { console.info('warning: fd_seek called, function not implemented'); return 0 },
    proc_exit: function () { console.info('warning: proc_exit called, function not implemented'); return },
    fd_fdstat_get: function () { console.info('warning: fd_fdstat_get called, function not implemented'); return 0 },
  }
}).then(
    (results) => {
      globalThis.results = results;

      console.time('wasm code')
      results.instance.exports._Z3runv();
      console.timeEnd('wasm code')

      const resultAddress = results.instance.exports._Z10get_bufferv();
      const width = results.instance.exports._Z9get_widthv();
      const height = results.instance.exports._Z10get_heightv();

      const memoryView = new Uint8ClampedArray(results.instance.exports.memory.buffer, resultAddress, width * height * 4)
      postMessage({
        kind: 'IMAGE_DATA',
        width,
        height,
        memoryView,
      })
    },
  );

In the wasi_snapshot_preview1, I input many functions required by the C runtime. Because I was very certain that the code wouldn’t use these functions, I just filled it with a bunch of garbage. Also, in the env, I added a function _Z13report_numberi. The logic it executes is to obtain the canvas’s width and height from various C functions and arguments, and pass the drawing result back to our UI thread via the postMessage function. Since the execution of WebAssembly will also block other codes, it’s clear that this piece of JavaScript code is executed in a worker.

Finally, we just need to prepare an entry HTML file, and we are all done.

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
  </head>
  <body>
    <canvas width="256" height="256" style="width: 256; height: 256;"></canvas>
    <script>
    const worker = new Worker("worker.js");
    worker.onmessage = (e) => {
      if (e.data.kind !== 'IMAGE_DATA') return;
      const imageData = new ImageData(e.data.memoryView, e.data.width, e.data.height, {});
      const canvas = document.querySelector('canvas')
      canvas.width = e.data.width;
      canvas.height = e.data.height;
      const ctx = canvas.getContext('2d');

      ctx.putImageData(imageData, 0, 0)
    }
    </script>
  </body>
</html>

Try It Out Yourself

I’ve also put the artifacts on my own blog; you can directly open this link to check out the result!

Play Helldivers 2 remotely with Moonlight and USB/IP project

Sun, 18 Feb 2024 00:00:00 +0000

The renowned friendship-ending PvE game “Helldivers 2” was recently launched, and as a helldivers 1 player, I eagerly welcomed it with open arms. After playing it for a few days, my impression is that while it has quite lots of bugs, it doesn’t have any major issues, and remains as authentic and true to its roots as ever.

However, on February 16th, when I launched the game from within the Moonlight streaming desktop as usual, all my inputs ceased to work: there was no response to any keyboard or mouse events. I ran over to check the computer in the cabinet and did not find any issues. After closing the game, the keyboard and mouse in Moonlight started working again, so I could only assume that it was the game’s anti-cheat mechanism that blocked the input events from Sunshine during streaming. Later, I also found on Reddit that other players had encountered the same problem:

Has anyone figured out how to get Helldivers 2 to work yet?

As a dedicated streaming user, I certainly can’t give up gaming over this minor obstacle, so I did some research and discovered quite a few solutions for implementing USB over IP/Ethernet:

VirtualHere is a commercial software that offers a one-stop solution for remote USB connection. However, this software requires a payment of $45 to unlock the connection of multiple devices. Additionally, the purchased license can only be used on one device.
USB/IP Project is an open-source initiative that has been around for some time and does not have the restrictions of the aforementioned commercial software.

Since I really didn’t want to spend the $45 (if it had been a floating license, I might have actually paid, as it seemed quite convenient upon trial), I dug out my long-unused Raspberry Pi. After updating the software repositories, all I needed to do was execute sudo apt install usbip to complete the installation of the server-side software. To conveniently run usbipd in the background, we can prepare a systemd unit like this:

[Unit]
Description=usbip host daemon
After=network.target

[Service]
Type=forking
ExecStartPre=/sbin/modprobe usbip-host
ExecStart=/usr/sbin/usbipd -D

[Install]
WantedBy=multi-user.target

After starting this daemon, you can view the USB devices connected to the current device by using the command sudo usbip list -l:

1
2
3

% sudo usbip list -l
 - busid 1-1.3 (046d:c52f)
   Logitech, Inc. : Nano Receiver (046d:c52f)

We can then use a command to make the related kernel module take control of this USB device; taking the example of the mouse receiver above: sudo usbip bind -b 1-1.3.

Next, go to the Windows PC to install the necessary software and drivers for the Client. This can be done by referring directly to the description in the README of the USB/IP for Windows repository; I will not go into further detail here.

After completing the driver installation, we only need to execute the command usbip.exe attach -r raspberrypi.local -b 1-1.3 to directly connect the mouse receiver to the Windows gaming machine. The same method can be applied to continue connecting other devices you want, such as a USB keyboard and gamepad. This way, we can happily engage in gaming.

Use SmartDNS as DNS Server on RouterOS

Sat, 26 Aug 2023 00:00:00 +0000

SmartDNS is a feature-rich local DNS server. You can check out the specific features it supports on their official website. It supports both Linux and Windows platforms.

Although RouterOS cannot directly run Linux software, starting from RouterOS 7.4beta4, MikroTik has added support for containers on certain modern architectures (arm, arm64, x86). With this mechanism, we can efficiently run some Linux applications directly on RouterOS devices, such as the SmartDNS we are going to demonstrate here.

Enable Container mode

For security reasons, RouterOS does not have container support enabled by default. We need to enable it with the following command:

`1`	`/system/device-mode/update container=yes`

After completion, the output will remind the user to restart the device. For example, for the CCR2004, simply press the reset button on the device. For CHR, performing a cold restart will achieve the same result. After the restart is complete, you can use the command /container/print to check if container support has been enabled.

Prepare Network Interfaces

In the official RouterOS documentation, veth is used to provide network access to containers. If you have experience with Linux’s network namespace (netns), you may already be familiar with the concept of veth. If you haven’t used it before, there’s no need to worry. The Linux official manual describes it as follows:

So, you can think of veth as a pair of network interfaces that are directly connected with an Ethernet cable. One end is “plugged into” your device, and the other end is “plugged into” the container or another network namespace. In essence, it creates a virtual Layer 2 point-to-point connection on your device.

RouterOS and Linux’s netns are essentially the same (given that RouterOS uses the Linux kernel). The main difference lies in the processes for creating and using them. To create a veth interface, you can use the following command:

`1`	`/interface/veth/add name=ether-pihole address=192.168.1.2/24 gateway=192.168.1.1`

Among these, address is the IP address of the interface in the container, and gateway is the default route for the container, which is typically the IP address of your RouterOS device.

Because I want devices in my local network to have direct access to smartdns, I simply add ether-pihole to the LAN bridge. This way, I won’t need any additional port forwarding or similar operations:

`1`	`/interface/bridge/port add bridge=bridge-lan interface=ether-pihole`

This way, our containers are in the same Layer 2 network as our local area network.

Create containers and configure services.

To avoid putting too much strain on the router’s NAND from application logs and image pulls within containers, you can create a RAM disk to store these files:

1
2

/disk/add type=tmpfs tmpfs-max-size=500M
/container/mounts/add name="smartdns-tmpfs" src="/tmp1/smartdns_log" dst="/var/log"

At the same time, for the convenience of making some custom adjustments to the network later on, I don’t intend to use ready-made Dockerfile from the internet to create our SmartDNS container. Instead, I will use alpine:latest as the base image and directly enter this container to download SmartDNS:

1
2

/container/config/set registry-url=https://registry-1.docker.io tmpdir=tmp1/pull
/container/add remote-image="library/alpine:latest" interface=ether-smartdns cmd="tail -f /dev/null" root-dir=container/smartdns mounts=smartdns-tmpfs dns=223.5.5.5 hostname="smartdns" logging=yes start-on-boot=yes

The ‘cmd’ written here is a command that can run for a long time without exiting on its own. This way, we have plenty of time to complete the container configuration. After running the create command, you can use ‘/container/print’ to observe the status. Once it’s completed, you can use ‘/container/start 0’ to start the container.

[admin@shanghai-router] > /container/shell 0
/ # ls
bin     dev     etc     home    lib     media   mnt     opt
proc    root    run     sbin    srv     sys     tmp     usr
var
/ #

At this point, we can download and install SmartDNS just like we would in a regular Linux environment:

# Download the smartdns binary
cd /usr/local/bin
wget https://github.com/pymumu/smartdns/releases/download/Release43/smartdns-aarch64 -O smartdns
chmod +x smartdns

# Create and edit configuration files
mkdir -p /etc/smartdns
vi /etc/smartdns/smartdns.conf

I won’t go into specific configuration details here. You can refer to the official documentation for that. After configuring it, you can directly start SmartDNS in the container to verify if the configuration is correct. Once everything is working fine, you can prepare to create your startup script. Since my container is directly exposed on the home LAN’s ethernet layer, I want to add an IPv6 address to the container directly. This will be distributed through SLAAC and DHCPv4 for IPv4/v6 configurations. My approach is straightforward: I’ll prepare a /start.sh file to handle the necessary configuration.

1
2
3

ip addr add 192.168.1.2/24 dev eth0 || true
ip addr add 2001:db8::1000/64 dev eth0 || true
/usr/local/bin/smartdns -f -p /tmp/smartdns.pid

Then update the container configuration and restart the container.

1
2
3

/container/set cmd="sh /start.sh" numbers=0
/container/stop 0
/container/start 0

Finally, you can use another device to verify the DNS service.

ntzyz@localhost ~ % dig @192.168.1.2 twitter.com

; <<>> DiG 9.10.6 <<>> @192.168.1.2 twitter.com
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 21559
;; flags: qr rd ra; QUERY: 1, ANSWER: 4, AUTHORITY: 0, ADDITIONAL: 0

;; QUESTION SECTION:
;twitter.com.			IN	A

;; ANSWER SECTION:
twitter.com.		597	IN	A	104.244.42.193
twitter.com.		597	IN	A	104.244.42.129
twitter.com.		597	IN	A	104.244.42.65
twitter.com.		597	IN	A	104.244.42.1

;; Query time: 33 msec
;; SERVER: 192.168.1.2#53(192.168.1.2)
;; WHEN: Sat Aug 26 14:45:00 CST 2023
;; MSG SIZE  rcvd: 93

In summary, using similar methods, you can run almost any Linux application to expand the functionality of RouterOS, such as BIND, pi-hole or even BIRD, and more.

Using NETMAP to obtain persistent IPv6 addresses and control source address

Fri, 18 Aug 2023 00:00:00 +0000

IPv6 addresses assigned by ISPs are usually not static:

[admin@shanghai-router] > /ipv6/dhcp-client/print
Columns: INTERFACE, STATUS, REQUEST, PREFIX
# INTERFACE          STATUS  REQUEST  PREFIX
0 pppoe-chinanet     bound   prefix   240e:388:6501:XX00::/56, 40m57s
1 pppoe-chinamobile  bound   prefix   2409:8a1e:6a64:2XX0::/60, 2d20h10m

While the address won’t change during the PPPoE session’s lifespan, ISPs often have mechanisms like timed interruptions and reconnections on their BRAS, leading to changes in our IPv6 address. In newer versions of RouterOS, when the DHCPv6-acquired prefix becomes invalid, the device updates the address “lifetime” to 0, thereby removing the old address. This is a good solution.

However, in this article about optimizing multi-line internet access using BGP full tables on RouterOS, it’s mentioned:

So, having a stable prefix alone isn’t sufficient; we need the device’s firewall to “intelligently” update the outgoing IPv6 source address to the one we want it to use. This is crucial to achieve optimal results for multi-ISP optimization. To achieve this goal, we can use the NETMAP target provided by netfilter.

Let’s assume our router has the following prefixes:

From China Telecom: 240e:388:6501:ab00::/56
From China Mobile: 2409:8a1e:6a64:2XX0::/60
Self-announced: 2602:feda:de0:700::/56

We’re selecting a fixed address range with the smallest prefix for translation, which is 2602:feda:de0:700::/60.

For outgoing connections to the pppoe-chinanet interface, where the source address is within the range of 2602:feda:de0:700::/60, we’ll use the firewall to map the source address to the range of 240e:388:6501:ab00::/60 (which is China Telecom’s address range):

`1`	`/ipv6/firewall/nat add chain=srcnat out-interface=pppoe-chinanet src-address=2602:feda:de0:700::/60 action=netmap to-address=240e:388:6501:ab00::/60`

Of course, we can’t just handle outbound traffic and ignore incoming traffic. Otherwise, when responses from external services reach the router and it realizes there’s no directly adjacent address using the 240e:388:6501:ab00::/60 prefix, we need to handle the return traffic:

`1`	`/ipv6/firewall/nat add chain=dstnat in-interface=pppoe-chinanet dst-address=240e:388:6501:ab00::/60 action=netmap to-address=2602:feda:de0:700::/60`

Similarly, rules for China Mobile can be written as follows:

1
2
3

/ipv6/firewall/nat
add chain=srcnat out-interface=pppoe-chinamobile src-address=2602:feda:de0:700::/60 action=netmap to-address=2409:8a1e:6a64:2XX0::/60
add chain=dstnat in-interface=pppoe-chinamobile dst-address=2409:8a1e:6a64:2XX0::/60 action=netmap to-address=2602:feda:de0:700::/60

Finally, we also need to add an update script to the corresponding DHCPv6 configuration to update the firewall rules, using China Mobile’s client as an example:

:log info ("dhcpv6 up script started");

:local  [/ipv6/pool/get [find name="ipv6-chinamobile-pd"] prefix];
:log info "prefix assigned by China Mobile is: $china_mobile_prefix";
:local china_mobile_new_prefix [:pick "$china_mobile_prefix" 0 ([:len "$china_mobile_prefix"] - 3)]

/ipv6/firewall/nat/set dst-address="$china_mobile_new_prefix/60" [find chain=dstnat action=netmap in-interface="pppoe-chinamobile"];
/ipv6/firewall/nat/set to-address="$china_mobile_new_prefix/60" [find chain=srcnat action=netmap out-interface="pppoe-chinamobile"];

:log info "netmap update okay";

That’s it!

We can pick an IPv6 address and try a traceroute:

ntzyz@nas-v2 ~ % curl ipv6.ip.sb
2406:4440:10:<redacted>
ntzyz@nas-v2 ~ % mtr 240e:f7:e01f:f1::30 --report -z
Start: 2023-08-18T13:58:52+0800
HOST: nas-v2                      Loss%   Snt   Last   Avg  Best  Wrst StDev
  1. AS142641 <redacted>           0.0%    10    0.3   0.4   0.3   0.4   0.0
  2. AS24400  2409:801e:f0:1::2fb 10.0%    10    4.3   4.9   3.9  10.5   2.1
  3. AS24400  2409:801e:f0:1::2fa  0.0%    10    4.2   4.3   3.9   5.2   0.4
  4. AS9808   2409:8080:0:2:206:2  0.0%    10    4.7   4.7   4.5   4.8   0.1
  5. AS9808   2409:8080:0:1:206:1  0.0%    10   21.8  16.4   7.9  25.4   6.6
  6. AS9808   2409:8080:0:1:1106: 70.0%    10   10.2  10.2  10.2  10.3   0.0
  7. AS9808   2409:8080:0:3:11e2:  0.0%    10   13.8  13.7  13.1  14.0   0.2
  8. AS4134   240e::c:b:5100:302   0.0%    10    8.5   8.5   8.1   8.7   0.2
  9. AS136190 240e:1c:b111:1fff:: 10.0%    10   12.2  15.3  11.1  43.4  10.5
 10. AS136190 240e:1c:b112:1122::  0.0%    10   11.9  11.9  11.8  12.0   0.1
 11. AS136190 240e:f7:e01f:ff::11  0.0%    10   13.0  14.7  12.6  27.2   4.5
 12. AS136190 240e:f7:e01f:f1::30  0.0%    10   10.6  11.5  10.6  11.8   0.3

As we can see, when this NAS accesses Cloudflare, it uses its my own prefix. When accessing domestic addresses, it actually uses China Telecom’s prefix.

Use Nix as Package Manager in macOS

Sun, 13 Aug 2023 00:00:00 +0000

Why Not Use Brew

Currently, the mainstream package management solution on macOS is still Homebrew, but I find it not very user-friendly for me for several reasons:

It’s really slow: Installing new packages is typically a low-frequency operation, yet Homebrew insists on checking for updates before installation. Even if you skip this step, the time difference required to install a package compared to other package managers is significant. I can’t quite comprehend why Brew takes so long for such a simple task.
“Infect” system files: It’s well-known that Homebrew installs everything into /usr/local/bin, making it easy for certain packages to maliciously integrate into the system’s default PATH. The community-driven nature of Brew’s operation can’t guarantee the safety of all packages, not to mention the potential for supply chain attacks.
Confusing in terms: Homebrew uses a plethora of brewing-related terms: Formula, Keg, Tap, Cellar, Bottle, Cask, etc. For someone like me who isn’t familiar with brewing, the learning curve is quite steep. I’d need to study these terms to understand what they all mean.

In summary, Brew isn’t a package manager that provides a comfortable experience for me. However, due to the necessity of working on macOS to make a living, I can’t just switch to Linux and start coding. Therefore, I’ve turned my attention to Nix.

Nix and NixOS

About a year ago, when I was struggling to manage more than 10 servers efficiently, I found NixOS, and there were many articles that detailed its features, which truly captivated me:

Centered around /etc/nixos: NixOS uses configuration files to control the software installed on the entire system and their configurations.
Reproducible Builds: With the same configuration files, NixOS consistently builds the same operating system.
Nix Is a Programable Package Manager: NixOS’s package manager configuration language, Nix, is Turing complete. By maintaining the configuration in a Git repository, I can share foundational configurations across all my servers, while also recording all historical versions for easy rollback and debugging.
Nix is a Purely Functional Language: Isn’t it fascinating?

Over time, I gradually replaced most of my Debian-based servers with NixOS. During this period, I noticed on NixOS’s download page that Nix (the package manager) has a macOS version available:

So, I made a plan to completely remove Brew from my system in the future.

“Trigger” for the Replacement

In my company, there’s a policy that allows for a replacement after three years when purchasing MacBook products. You can even choose to repurchase the work laptop. As soon as the three years were up, I initiated the end-of-term replacement process with the IT department. Although this Intel MacBook Pro was configured with an i7 and 32GB of RAM, it was still frustratingly slow and on the verge of mental crashing while coding.

On the day of application, IT informed me that I could come and pick up the new laptop. The replacement was another 16-inch MacBook, equipped with an M2 processor and 32GB of memory, which perfectly met my expectations. As for software, I decided not to migrate data from the old Intel Mac. After all, the architecture transition made me a bit skeptical about its reliability. So this time, I didn’t install Brew, and I planned to give Nix a try outside of NixOS.

Installing nix

As shown in the screenshot from the official website above, installing Nix is as simple as running a curl command.

`1`	`sh <(curl -L https://nixos.org/nix/install)`

However, unlike most “one-key” scripts, Nix provides detailed output during the installation process, especially when it needs to use sudo:

Once the installation is complete, we can use the nix command in the terminal:

Nix has a distinct advantage over Brew: its installed files are placed in an APFS container, making the removal of Nix quite convenient—you just need to delete the container.

At this point, we have completed the installation of Nix, but it’s still a bit different from managing most software with a configuration.nix file as in NixOS.

Installing nix-darwin

nix-darwin brings the software management approach of NixOS to macOS. You can easily install nix-darwin using Nix:

1
2

nix-build https://github.com/LnL7/nix-darwin/archive/master.tar.gz -A installer
./result/bin/darwin-installer

After the installation, you’ll find the darwin-configuration.nix file in ~/.nixpkgs. This configuration file will be the key to managing your macOS system using nix-darwin:

{ config, pkgs, ... }:

{
  environment.systemPackages = [
    pkgs.vim
  ];

  services.nix-daemon.enable = true;
  programs.zsh.enable = true;  # default shell on catalina
  system.stateVersion = 4;
}

As I am a frontend developer, I need to install node. Additionally, I’m more accustomed to using GNU’s coreutils rather than BSD’s, so I have adjusted the configuration as follows:

{ config, pkgs, lib, ... }:

{
  nixpkgs.config.allowUnfreePredicate = pkg: builtins.elem (lib.getName pkg) [
    "unrar"
  ];

  environment.systemPackages = with pkgs; [
    vim ffmpeg-full coreutils gnugrep gnused gawk htop mtr
    smartmontools neofetch rsync p7zip hugo ncdu
    ipmitool iperf3 wireguard-tools jq p7zip unrar
  ];

  services.nix-daemon.enable = true;
  programs.zsh.enable = true;
  system.stateVersion = 4;
}

Afterward, you only need to run darwin-rebuild switch to allow darwin-nix to complete the installation and configuration of software (I haven’t used the configuration capabilities here due to personal preference). Once this is done, all the software will be available in the PATH:

Looking at the logs, it’s easy to notice the existence of the /Applications/Nix Apps directory. In fact, it’s not just command-line software; GUI software can also be installed through Nix, such as this terminal emulator called kitty:

Optimizing multi-wan internet access using BGP full table on RouterOS.

Wed, 09 Aug 2023 00:00:00 +0000

Recently, I installed a 1000M Shanghai Mobile broadband in the place I am currently staying. In order to make full use of the new broadband and the existing Shanghai Telecom broadband, it’s necessary to use some magic to utilize both upstreams. Here, we will take the example of the CCR2004-16G-2S+ router to explain how to optimize network access using BGP full table.

Our target strategy is very clear:

For IP addresses targeting China Telecom (AS4134, AS4812), use China Telecom as default gateway.
For all remaining non-China ASNs, use a special gateway within the local network tunneling all traffic to one server located in Hong Kong.
The rest should go through China Mobile.

Basic network configuration

First, we need to connect our router to the internet. Assuming the China Telecom network is connected to ether1 and the China Mobile network is connected to ether2, we can create two PPPoE clients like this:

1
2

/interface/pppoe-client/add name="pppoe-chinanet" interface=ether1 profile=default add-default-route=no dial-on-demand=no use-peer-dns=no allow=pap,chap,mschap1,mschap2
/interface/pppoe-client/add name="pppoe-chinamobile" interface=ether2 profile=default add-default-route=no dial-on-demand=no use-peer-dns=no allow=pap,chap,mschap1,mschap2

The part about the PPPoE account and password is omitted above. You can fill it in as needed. It is worth noting that the newly created PPPoE connection does not use the default gateway. Here, we can add a static route as the default gateway configuration. Assuming we plan to use mobile broadband as the default upstream connection:

`1`	`/ip/route/add dst-address=0.0.0.0/0 routing-table=main gateway=pppoe-chinamobile distance=100`

At this point, our router should be able to access the network correctly. Let’s use traceroute to verify it.

[admin@shanghai-router] > /tool/traceroute 223.5.5.5
Columns: ADDRESS, LOSS, SENT, LAST, AVG, BEST, WORST, STD-DEV
#  ADDRESS         LOSS   SENT  LAST     AVG   BEST  WORST  STD-DEV
1  <redacted>      0%        9  3.9ms    4.7   3.6   8.9    1.9
2  <redacted>      75%       9  timeout  3.7   3.6   3.7    0.1
3  <redacted>      12.5%     8  timeout  3.7   3.6   3.8    0.1
4  <redacted>      0%        8  12.9ms   7.8   3.9   20.3   5.5
5                  100%      8  timeout
6  116.251.116.93  0%        8  6.2ms    12.9  5.6   38.3   10.6
7                  100%      8  timeout
8                  100%      8  timeout
9  223.5.5.5       0%        8  5.5ms    5.4   5.2   5.6    0.1

At the same time, we can also test the PPPoE connection of the China Telecom network by specifying the interface in parameters:

[admin@shanghai-router] > /tool/traceroute 223.5.5.5 interface=pppoe-chinanet
Columns: ADDRESS, LOSS, SENT, LAST, AVG, BEST, WORST, STD-DEV
 #  ADDRESS         LOSS   SENT  LAST     AVG   BEST  WORST  STD-DEV
 1  <redacted>      7.7%     13  3.8ms    4.7   1.8   7.7    1.8
 2  <redacted>      0%       13  10.3ms   8.7   4.6   18.4   3.2
 3  <redacted>      66.7%    13  timeout  6.6   3.9   14.2   4.4
 4  <redacted>      66.7%    12  timeout  5.5   3.8   8.7    1.9
 5  <redacted>      0%       12  3.2ms    6.3   3.1   25.3   6.7
 6  <redacted>      0%       12  5.8ms    18.2  5.5   150.8  40
 7                  100%     12  timeout
 8                  100%     12  timeout
 9                  100%     12  timeout
10  223.5.5.5       0%       12  5.4ms    5.5   5.3   5.8    0.2

Of course, we also need to create some NAT rules in /ip/firewall/nat, so that devices in our LAN can also access the internet. This part is relatively basic, so I won’t go into detail.

Obtain realtime BGP full routing table

If you are a BGP player, you can totally skip this part, but if you are not familiar with BGP or don’t have an ASN, you may encounter some difficulties. Here, you can acquire the complete BGP routing table by purchasing the cheapest Vultr server, and it does not require you to have your own ASN. It offers great value for money. There are many online resources available that provide step-by-step instructions on how to perform the related operations, you can google it by yourself.

After completing the operations on Vultr, we can follow the official guide and establish a BGP session with Vultr to obtain full routing table data. It is very easy. The key configurations are as follows:

protocol device {}

protocol kernel kernel4 {
  ipv4 {
    import none;
    export where source != RTS_DEVICE && proto != "bgp_vultr_v4";
  };
}

protocol kernel kernel6 {
  ipv6 {
    import none;
    export filter {
      if source = RTS_DEVICE then reject;
      if proto = "bgp_vultr_v4" then reject;
      reject;
    };
  };
}

protocol static static_bgp_neighbor_v6 {
  ipv6;
  route 2001:19f0:ffff::1/128 via fe80::fc00:4ff:fe08:ee5f % enp1s0;
}

protocol static static_bgp_neighbor_v4 {
  ipv4;
  route 169.254.169.254/32 via a.b.c.1 % enp1s0;
}

protocol bgp bgp_vultr_v4 {
  local as 142641;
  neighbor 169.254.169.254 as 64515;
  source address a.b.c.d;
  multihop 2;
  password "xxxxxxxx";

  ipv4 {
    import all;
    export none;
  };
}

protocol bgp bgp_vultr_v6 {
  local as 142641;
  neighbor 2001:19f0:ffff::1 as 64515;
  source address 2001:db8::1;
  multihop 2;
  password "xxxxxxx";

  ipv6 {
    import all;
    export none;
  };
}

If everything goes well, we can use the command show protocol in birdc to see that our session status is already “Established”.

1
2
3

Name         Proto      Table      State  Since         Info
bgp_vultr_v4 BGP        ---        up     2023-01-21    Established
bgp_vultr_v6 BGP        ---        up     2023-01-21    Established

We can also use the “show route for” command to view specific forwarding rule, for example:

bird> show route for 1.1.1.1 all
Table master4:
1.1.1.0/24           unicast [bgp_vultr_v4 2023-08-01 from 169.254.169.254] * (100/?) [AS13335i]
	via 45.32.28.1 on enp1s0
	Type: BGP univ
	BGP.origin: IGP
	BGP.as_path: 64515 20473 13335
	BGP.next_hop: 169.254.169.254
	BGP.local_pref: 100
	BGP.aggregator: 10.34.7.24 AS13335
	BGP.community: (20473,200) (64515,44)
	BGP.large_community: (20473, 200, 13335) (20473, 200, 45686)

Thus we are now ready with our own BGP speaker.

Establish a tunnel between the BGP feeder and the router.

Home broadband usually does not have a static IP address. To minimize configuration changes, the simplest way is to set up a tunnel between the router at home and a server with full BGP table, and WireGuard can be used for this purpose. Its advantage is that it has native support in newer kernels and RouterOS. Additionally, it uses UDP protocol, which makes it convenient to use various IEPL/IPLC forwarding services to improve reliability.

On the feeder, let’s create a new VPN interface with the following configuration:

[Interface]
PrivateKey = xxx
Address = 10.22.33.1/24, fd34:38a:f295::1/64
MTU = 1400
ListenPort = 114514

[Peer]
PublicKey = yyy
AllowedIPs = 10.22.33.2/32, fd34:38a:f295::2/128

Then, create a WireGuard interface on RouterOS and peering it with our BGP speaker:

/interface/wireguard/add name="wg-bgp-speaker" private-key="xxx" mtu=1400
/interface/wireguard/peers/add endpoint-address=172.16.1.1 endpoint-port=3000 interface=wg-bgp-speaker public-key="qzgKhWy8u0DBvXcrhfFDYPwKzKipWzCsxR5ecNIYZFs=" persistent-keepalive=60 allowed-address=10.22.33.1/32,fd34:38a:f295::1/128

/ip/address/add interface=wg-bgp-speaker address=10.22.33.2/24
/ipv6/address/add interface=wg-bgp-speaker address=fd34:38a:f295::2/64

So we have created a direct connected tunnel, which can only allow our router and server to communicate point-to-point. But that’s enough.

Prepare the routing filters

According to the rules mentioned earlier, we can prepare the routing filters on the RouterOS side. In this filter, we directly choose the gateway used by this route based on the end of the AS path. Taking IPv4 as an example:

if (dst-len == 0) {
  reject
}

set distance 50;

if (not bgp-as-path [[:china_asn_list:]]$) {
  set gw 198.18.1.1;
  set gw-interface ipip-internet-proxy;

  accept;
} else {
  if (bgp-as-path 4134$|4812$) {
    set gw %pppoe-chinanet;
  } else {
    set gw %pppoe-chinamobile;
  }
}

accept;

There are two things that need clarification:

china_asn_list is an ASN (Autonomous System Number) list obtained from GitHub, which is used in mainland China. It can be maintained in /routing/filter/num-list, for example, by adding AS4134: /routing/filter/num-list/add list=china_asn_list range=4134
You can directly set an interface as a gateway using set gw %interface command, which allows the kernel to pay attention to which next hop to use, and it can reduce a lot of maintenance costs.

For IPv6 filter, the content remains largely the same, with only a few adjustments required for the cross-border access gateway.

Once these two filters are prepared, the final step is to bring the full table home.

Bring full-table home

The final task is to establish a BGP session between our feeder and router. First, we need to add two new protocols on the speaker.

protocol bgp bgp_speaker_v4 {
  local as 65000;
  neighbor 10.22.33.2 as 65001;
  source address 10.22.33.1;
  passive;
  ipv4 {
    import none;
    export where proto = "bgp_vultr_v4" && 223.5.5.5 !~ net;
  };
};

protocol bgp bgp_speaker_v6 {
  local as 65000;
  neighbor fd34:38a:f295::2 as 65001;
  source address fd34:38a:f295::1;
  passive;
  multihop;
  ipv6 {
    import none;
    export where proto = "bgp_vultr_v6";
  };
};

Then add the corresponding BGP connection on RouterOS:

/routing/bgp/connection/add remote.address=10.22.33.1/32 remote.as=65000
  local.address=10.22.33.2 remote.role=ebgp
  routing-table=main router-id=10.22.33.2 as=65001 multihop=yes hold-time=infinity address-families=ip
  output.filter-chain=bgp-speaker-export
  input.filter=bgp-speaker-v4

/routing/bgp/connection/add remote.address=fd34:38a:f295::1/128 .as=65000
  local.address=fd34:38a:f295::2 .role=ebgp
  routing-table=main router-id=10.22.33.2 as=65001 multihop=yes hold-time=infinity address-families=ipv6
  output.filter-chain=bgp-speaker-export
  input.filter=bgp-speaker-v6

After waiting for the entire table to enter FIB, we can start verifying the effectiveness of our traffic diversion. We will use a China Telecom/China Mobile/China Unicom/Cloudflare addresses respectively for testing.

# China Telecom
[admin@shanghai-router] > /ip/route/print where 221.227.153.1 in dst-address and routing-table=main and !static
Flags: D - DYNAMIC; A - ACTIVE; b, y - BGP-MPLS-VPN
Columns: DST-ADDRESS, GATEWAY, DISTANCE
    DST-ADDRESS     GATEWAY         DISTANCE
DAb 221.224.0.0/13  pppoe-chinanet        50

# China Mobile
[admin@shanghai-router] > /ip/route/print where 120.232.236.5 in dst-address and routing-table=main and !static
Flags: D - DYNAMIC; A - ACTIVE; b, y - BGP-MPLS-VPN
Columns: DST-ADDRESS, GATEWAY, DISTANCE
    DST-ADDRESS       GATEWAY            DISTANCE
DAb 120.192.0.0/10    pppoe-chinamobile        50
DAb 120.232.0.0/16    pppoe-chinamobile        50
DAb 120.232.236.0/22  pppoe-chinamobile        50

# China Unicom
[admin@shanghai-router] > /ip/route/print where 103.45.78.1 in dst-address and routing-table=main and !static
Flags: D - DYNAMIC; A - ACTIVE; b, y - BGP-MPLS-VPN
Columns: DST-ADDRESS, GATEWAY, DISTANCE
    DST-ADDRESS     GATEWAY            DISTANCE
DAb 103.45.76.0/22  pppoe-chinamobile        50

# Cloudflare
[admin@shanghai-router] > /ip/route/print where 1.1.1.1 in dst-address and routing-table=main and !static
Flags: D - DYNAMIC; A - ACTIVE; b, y - BGP-MPLS-VPN
Columns: DST-ADDRESS, GATEWAY, DISTANCE
    DST-ADDRESS  GATEWAY                         DISTANCE
DAb 1.1.1.0/24   198.18.1.1%ipip-internet-proxy        50

In this way, our dual-line broadband IPv4 is fully utilized, although v6 achieves traffic diversion through these methods, there are still issues because there is no NAT and devices cannot intelligently choose which prefix of the IP address to use. The subsequent article will introduce how to solve this problem. Finally, let’s show the results:

Hands on MikroTik RB4011iGS+RM

Mon, 05 Jun 2023 00:00:00 +0000

The router is a second-hand product that I purchased from XianYu (Chinese clone of eBay) on April 18, 2023. The specifications are roughly as follow:

Product code                 RB4011iGS+RM
Architecture                 ARM 32bit
CPU                          AL21400
CPU core count               4
CPU nominal frequency        auto (533 - 1900) MHz
Switch chip model            RTL8367SB
Dimensions                   228 x 120 x 30 mm
RouterOS license             5
Operating System             RouterOS (v7 only)
Size of RAM                  1 GB
Storage size                 512 MB
Storage type                 NAND
MTBF                         Approximately 200'000 hours at 25C
Tested ambient temperature   -40°C to 70°C
IPsec hardware acceleration  Yes
Suggested price              $219.00

The reason for buying such a router is simple: I’ve been using virtualized Linux routers before, which have many advantages like outstanding performance and flexible. However, maintaining the hypervisor is very difficult, any hardware adjustments can cause the entire network to go down, which can annoy the roommates if it happens frequently. If I want to use Proxmox’s HA for hot migration and high availability, I’d need to set up a Ceph storage service and two new EPYC servers, which would be too expensive in terms of hardware and running costs, in that case it’s better to get a separate 1U Celeron server as a router.

Hardware requirements

I myself haven’t paid much attention to consumer-grade wireless routers for a while, so I won’t make any horizontal comparisons for now. As for my own needs, besides the standard RJ45 GbE interface, I also need at least one SFP+ cage for future-proofing. ISPs in Shanghai have basically rolled out 1000Mbps broadband, and considering the low update frequency of routers, it shouldn’t be too extravagant to buy a router that supports 10Gbps interfaces in preparation for future gigabit or higher broadband.

This device supports 10 Gigabit Ethernet ports and one SFP+ cage, which meets my needs perfectly. It is also equipped with a sufficiently powerful 1.9GHz ARM processor that can handle my requirement of running 500Mbps on WireGuard.

But it should be noted that the 10 Ethernet interfaces of this router are not all directly connected to the switch chip/CPU. Instead, they are divided into two groups, each connected to a 2.5G switch chip and then connected to the CPU. You can refer to this diagram:

In other words, when connecting devices, it is necessary to avoid accessing large flow across switches as much as possible, otherwise it may hit the 2.5Gbps communication limit of the switch chip. However, this issue can be ignored for me. My anticipated usage scenario is to connect the ISP cable to ether1 and the ZTE switch to SFP+. The local network traffic will not pass through the router, so it will not hit this limit at all.

Software Configuration

Here things here are starting to get messy, because my needs are really messy.

BGP support is required: I have my own Autonomous System Number (ASN) and a publicly reachable IPv6 prefix. I need to establish BGP sessions and advertise them in Hong Kong and to upstream providers. Each router in different regions will use private ASN in a large internal network for routing and BGP will be used as the routing protocol.
WireGuard support is required: My internal network primarily uses WireGuard for communication. It would be inconvenient if this device does not support WireGuard (IPsec and OpenVPN are not as user-friendly as WireGuard, and I believe most people would agree on this).
Performance should be sufficient to accept a full BGP routing table from external sources for proxy routing: This is well understood, older devices may not be able to handle this.
Basic features such as VLAN, VRF, and policy-based routing should also be supported.

Fortunately, RouterOS v7 can basically meet all the above requirements.

Upstream Network Configuration

Let’s create a PPPoE client for internet first:

[admin@MikroTik] > /interface/pppoe-client/print detail
Flags: X - disabled, I - invalid; R - running
 0  R name="pppoe-chinanet" max-mtu=auto max-mru=auto mrru=disabled
      interface=ether1 user="redacted" password="redacted" profile=default
      keepalive-timeout=60 service-name="" ac-name="" add-default-route=yes
      default-route-distance=100 dial-on-demand=no use-peer-dns=no
      allow=pap,chap,mschap1,mschap2

Then create one IPv6 DHCPv6 client to obtain IPv6 prefixes from ChinaNet:

[admin@MikroTik] > /ipv6/dhcp-client/print detail
Flags: D - dynamic; X - disabled, I - invalid
 0    ;;; ChinaNet DHCPv6 PD
      interface=pppoe-chinanet status=bound duid="redacted"
      dhcp-server-v6=fe80::ce1a:faff:feeb:d1c0 request=prefix
      add-default-route=no use-peer-dns=no dhcp-options=""
      pool-name="chinanet-ipv6-pd" pool-prefix-length=56 prefix-hint=::/0
      script=redacted
      dhcp-options="" prefix=240e:redacted::/56, 32m8s

Thus the basic internet access is almost okay.

WireGuard Configuration

The configuration style of WireGuard in RouterOS is different from other systems, they separates the configuration of interfaces and peers. When adding peers, you need to explicitly specify an interface.

[admin@MikroTik] /interface/wireguard> print detail where name=sddn-unicom-cz
Flags: X - disabled; R - running
 2  R name="sddn-unicom-cz" mtu=1400 listen-port=redacted
      private-key="redacted"
      public-key="redacted"

[admin@MikroTik] /interface/wireguard> peers print detail where interface=sddn-unicom-cz
Flags: X - disabled
 0   interface=sddn-unicom-cz
     public-key="redacted"
     endpoint-address=redacted endpoint-port=redacted
     current-endpoint-address=redacted current-endpoint-port=redacted
     allowed-address=0.0.0.0/0,::/0 persistent-keepalive=25s rx=830.5MiB
     tx=507.9MiB last-handshake=1m5s

Since the interface configuration is complete, we need to add v4 and v6 addresses to our interface configurations on /ip/address and /ipv6/address respectively. Please note that RouterOS is not very friendly when establishing BGP sessions using IPv4 addresses with /32 prefix. It’s recommended to use /31 or larger interface addresses for further BGP configurations.

[admin@MikroTik] /routing/bgp> /ip/address/print where interface=sddn-unicom-cz
Columns: ADDRESS, NETWORK, INTERFACE
# ADDRESS       NETWORK    INTERFACE
;;; IPv4 Address for BGP Session with Unicom-Changzhou
2 28.0.15.9/30  28.0.15.8  sddn-unicom-cz

[admin@MikroTik] /routing/bgp> /ipv6/address/print where interface=sddn-unicom-cz
Flags: D - DYNAMIC; G, L - LINK-LOCAL
Columns: ADDRESS, INTERFACE, ADVERTISE
#    ADDRESS                      INTERFACE       ADVERTISE
;;; IPv6 Address for BGP Session with Unicom-Changzhou
1  G 2001:db8:reda:cted::/126     sddn-unicom-cz  no
2 DL fe80::6b4e:4fe3:b34:cf1c/64  sddn-unicom-cz  no

BGP session for intranet

Just like using BIRD, it is best for us to prepare a template first.

[admin@MikroTik] /routing/bgp> template/print
Flags: * - default; X - disabled, I - inactive
 0 *  name="default" routing-table=main router-id=28.1.1.1 as=4233331011
      multihop=no address-families=ip,ipv6
      output.redistribute=connected,static

Then create BGP configurations for v4 and v6 using these two templates.

[admin@MikroTik] /routing/bgp> connection/print where name="bgp-sddn-unicom-cz-v4" or name="bgp-sddn-unicom-cz-v6"
Flags: D - dynamic, X - disabled, I - inactive
 0   name="bgp-sddn-unicom-cz-v6"
     remote.address=2001:db8:redacted::a .as=4233330010
     local.address=2001:db8:redacted::9 .role=ebgp
     routing-table=main router-id=28.1.1.1 templates=default as=4233331011
     nexthop-choice=default multihop=no address-families=ipv6
     output.redistribute=connected,static,dhcp .filter-chain=sddn-export
     .no-early-cut=no .keep-sent-attributes=no
     input.filter=sddn-cn-import

 1   name="bgp-sddn-unicom-cz-v4"
     remote.address=28.0.15.10 .as=4233330010
     local.address=28.0.15.9 .role=ebgp
     routing-table=main router-id=28.1.1.1 templates=default as=4233331011
     multihop=no address-families=ip
     output.redistribute=connected,static .filter-chain=sddn-export
     input.filter=sddn-cn-import

Of course we may also need to add some filters to make some adjustments to the routing priority as needed.

[admin@MikroTik] /routing/bgp> ../filter/rule/print where chain=sddn-cn-import
Flags: X - disabled, I - inactive
 0   chain=sddn-cn-import
     rule="if (afi ipv4) {\r\n  set pref-src 28.0.99.11;\r\n}\r\n\r\nif (afi
     ipv6) {\r\n  set pref-src 2001:db8:redacted::28.0.99.11;\r\n}\r\n\r\nif
     (bgp-large-communities includes 142641:100:2) {\r\n  set distance
     +5;\r\n} else {\r\n  set distance -5;\r\n}\r\n\r\naccept"

In addition to the WireGuard + BGP connection to one of my servers mentioned above, I have also configured a connection to Hong Kong and Japan. I will not go into further details here.

Bring BGP full-table back to home

The easiest way to obtain a full table solution currently is still through Vultr’s cheapest VPS. There are many articles available online for reference, and those who need it will naturally proceed with the operation. However, my implementation strategy on RouterOS is rather mysterious: I use BGP to accept non-mainland IPv4 routes and mainland IPv6 routes.

Regarding the IPv4 part, I plan to use the gateway obtained through PPPoE to access the Internet directly. I want to pull non-mainland routing prefixes into the main table and set them up as an ipip tunnel interface within the internal network by using a filter rule. Since the gateway (peer) is dynamic after the PPPoE connection is established, I prefer to avoid using this parameter as much as possible.

[admin@MikroTik] /routing/bgp> ../filter/rule/print where chain=bgp-speaker-proxy-v4
Flags: X - disabled, I - inactive
 3   chain=bgp-speaker-proxy-v4
     rule="if (dst-len > 0) {\n  set gw 28.0.15.14;\n  set gw-interface ipip-
     internet-gateway;\n  accept;\n} else {\n  reject;\n}"

For IPv6, the router in my intranet has announced a default route, which is dynamic. Fortunately, the link-local addressing mechanism of IPv6 allows me to use a stable fe80::xxxx%pppoe-chinanet gateway as an exit, which is very fixed. That’s why I used the opposite mode compared to v4:

[admin@MikroTik] /routing/bgp> ../filter/rule/print where chain=bgp-speaker-direct-v6
Flags: X - disabled, I - inactive
 4   chain=bgp-speaker-direct-v6
     rule="if (dst-len > 0) {\r\n  set gw fe80::redacted;\r\n  set  gw-interface
     pppoe-chinanet;\r\n  set distance -5;\r\n  accept;\r\n} else {\r\n  reject;
     \r\n}"

Let’s test the routing effect of the router using 1.1.1.1, 223.5.5.5, 240e:96c:6100:b15:3e::, and 2001:470:0:1f9::2 separately:

[admin@MikroTik] /routing/bgp> /ip/route/print where 1.1.1.1 in dst-address and
routing-table=main
Flags: D - DYNAMIC; A - ACTIVE; b, v, y - BGP-MPLS-VPN
Columns: DST-ADDRESS, GATEWAY, DISTANCE
    DST-ADDRESS  GATEWAY                         DISTANCE
DAv 0.0.0.0/0    pppoe-chinanet                       100
DAb 1.1.1.0/24   28.0.15.14%ipip-internet-proxy        20

[admin@MikroTik] /routing/bgp> /ip/route/print where 223.5.5.5 in dst-address and routing-table=main
Flags: D - DYNAMIC; A - ACTIVE; v, y - BGP-MPLS-VPN
Columns: DST-ADDRESS, GATEWAY, DISTANCE
    DST-ADDRESS  GATEWAY         DISTANCE
DAv 0.0.0.0/0    pppoe-chinanet       100

[admin@MikroTik] /routing/bgp> /ipv6/route/print where 240e:96c:6100:b15:3e:: in dst-address and routing-table=main
Flags: D - DYNAMIC; A - ACTIVE; b, v, y - BGP-MPLS-VPN
Columns: DST-ADDRESS, GATEWAY, DISTANCE
    DST-ADDRESS    GATEWAY                                   DISTANCE
DAb ::/0           2001:db8:redacted:cc23::6                       20
D b ::/0           2001:db8:redacted:cc23::a                       25
D v ::/0           pppoe-chinanet                                 100
DAb 240e::/20      fe80::ce1a:faff:feeb:d1c0%pppoe-chinanet        15
DAb 240e:96c::/30  fe80::ce1a:faff:feeb:d1c0%pppoe-chinanet        15
DAb 240e:96c::/32  fe80::ce1a:faff:feeb:d1c0%pppoe-chinanet        15

[admin@MikroTik] /routing/bgp> /ipv6/route/print where 2001:470:0:1f9::2 in dst-address and routing-table=main
Flags: D - DYNAMIC; A - ACTIVE; b, v, y - BGP-MPLS-VPN
Columns: DST-ADDRESS, GATEWAY, DISTANCE
    DST-ADDRESS  GATEWAY                   DISTANCE
DAb ::/0         2001:db8:redacted:cc23::6       20
D b ::/0         2001:db8:redacted:cc23::a       25
D v ::/0         pppoe-chinanet                 100

It can be seen that basically, the expected correct routing is met.

IPv6 Prefix Address Translation

At this moment, my router has both an IPv6 prefix from China Telecom and an IPv6 prefix that I obtained from my own LIR. However, because the LAN side uses SLAAC to distribute its own prefix, even though I have performed traffic engineering using the complete table data, the source address still seems off: the Telecom interface receives a packet with an address that is completely different from the one it distributed. Interestingly, Telecom does not filter this type of traffic but correctly forwards the packet using its own network to the final destination. When the target server sends a response, it “conforms to expectations” by using the source address from the previous packet as the destination address. Since this address is advertised by my ASN in Hong Kong, it will be routed to routers in Hong Kong and then transmitted through my intranet to my home, resulting in poor network performance as the network packets travel globally.

The solution to this problem is quite simple, we just need to add a little magic to the IPv6 firewall and perform NAT for the traffic passing through the China Telecom gateway.

[admin@MikroTik] /routing/bgp> /ipv6/firewall/nat/print
Flags: X - disabled, I - invalid; D - dynamic
 0    chain=dstnat action=netmap to-address=2001:redacted::/56
      dst-address=240e:redacted::/56 in-interface=pppoe-chinanet log=no
      log-prefix=""

 1    chain=srcnat action=netmap to-address=240e:redacted::/56
      src-address=2001:redacted::/56 out-interface=pppoe-chinanet log=no
      log-prefix=""

Of course, because these rules are written with the prefix of the router’s telecommunication, we need to make some adjustments to the previous DHCP client. We can use a script to dynamically update these two firewall rules.

:log info ("dhcpv6 client script started");

:local ChinaNetPrefix [/ipv6/pool/get [find name="chinanet-ipv6-pd"] prefix];
:log info "prefix assigned by ChinaNet is: $ChinaNetPrefix";

:local ChinaNetPPPLinkLocalAddress [/ipv6/dhcp-client/get [find pool-name="chinanet-ipv6-pd"] dhcp-server-v6];
:log info "link local address ChinaNet is: $ChinaNetPPPLinkLocalAddress";

/routing/filter/rule/set rule="if (dst-len > 0) {set gw $ChinaNetPPPLinkLocalAddress;set gw-interface pppoe-chinanet;set distance -5;accept;} else {reject;}" [find chain="bgp-speaker-direct-v6"];
:log info "filter update okay";

:local ChinaNetNewPrefix [:pick "$ChinaNetPrefix" 0 ([:len "$ChinaNetPrefix"] - 3)]
/ipv6/firewall/nat/set dst-address="$ChinaNetNewPrefix/56" [find chain=dstnat action=netmap];
/ipv6/firewall/nat/set to-address="$ChinaNetNewPrefix/56" [find chain=srcnat action=netmap];

:log info "netmap update okay";

Multiple VLANs and Acceleration of Game/Steam downloads

The solution for these issues is relatively consistent, which is to establish multiple different routing tables, set up respective routes in the tables, and associate the configuration of each VLAN interface with these tables using /routing/rule.

[admin@MikroTik] /routing/bgp> /routing/rule/print
Flags: X - disabled, I - inactive
 0   src-address=10.0.0.0/24 interface=wg-as4134 action=lookup
     table=egress-pppoe

 1   interface=vlan-gaming-hk action=lookup table=gaming-hk

 2   interface=vlan-gaming-jp action=lookup table=gaming-jp

[admin@MikroTik] /routing/bgp> /ip/firewall/mangle/print where chain=prerouting

Flags: X - disabled, I - invalid; D - dynamic
 0  D ;;; special dummy rule to show fasttrack counters
      chain=prerouting action=passthrough

 5    ;;; allow Hong Kong gaming VLAN accessing other intranet
      chain=prerouting action=mark-routing new-routing-mark=main
      passthrough=no src-address=28.1.4.0/24 dst-address=28.0.0.0/8
      in-interface=vlan-gaming-hk log=no log-prefix=""

 6    ;;; allow Japan gaming VLAN accessing other intranet
      chain=prerouting action=mark-routing new-routing-mark=main
      passthrough=no src-address=28.1.5.0/24 dst-address=28.0.0.0/8
      in-interface=vlan-gaming-jp log=no log-prefix=""

 7    ;;; HKRPG overseas UDP
      chain=prerouting action=mark-routing new-routing-mark=gaming-jp
      passthrough=no protocol=udp dst-port=23301 log=no log-prefix=""

Some Other Issues Occurred

The implementation of ipip/gre tunnel in RouterOS refers to Cisco’s keep alive implementation. When used with Linux devices, it is unable to determine if the other party is online. You can check the specific reason here: Replying to GRE keep alive packets on Linux

The above is about the situation of using RB4011 in my own network environment. In actual use, there are hardly any problems except for occasional strange issues with RouterOS. Finally, here is a screenshot of WebFig:

My NAS and Data Backup

Wed, 15 Feb 2023 00:00:00 +0000

I had experienced some painful accidents where hard drives failed and caused data loss, but luckily I didn’t lose any crucial files. I couldn’t afford to make the same mistake again, so I decided to add a backup mechanism to my existing storage system, and it has been effective. Recently, I made some adjustments to my data backup plan, and I’ll briefly record it here.

To enable fast photo preview and image processing, I saved the complete photo catalog of CaptureOne on my Apple Silicon MacBook. In order to save storage space, I use my own script to convert all ARW files to Adobe DNG files before importing. Even just compressing them once with zip can save a significant amount of space. Storing data on a Mac is obviously risky, as we all know that since MacBook with T2 chip encounters a problem, the data is gone forever. It’s unrealistic to hope that this kind of hardware, which will eventually fail, won’t fail as late as possible. Therefore, I periodically copy my catalog to my NAS using rsync.

As for NAS, it is actually a common Debian Server that has three 6TB HGST Ultrastar HC310 drives directly connected. All drives are formatted as ext4 and then merged into an 18TB union filesystem using GitHub: mergerfs. Multiple directories are created within the merged filesystem to categorize and store files.

% mount
/dev/sdc1 on /media/HGST2 type ext4 (rw,relatime,data=ordered)
/dev/sdd1 on /media/HGST1 type ext4 (rw,relatime,data=ordered)
/dev/sde1 on /media/HGST0 type ext4 (rw,relatime,data=ordered)
0:1:2 on /srv/NAS type fuse.mergerfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other)

% ls /srv/NAS
ISCSI  REDACTED  NTZYZ  PUBLIC  TIME_MACHINE

Obviously, this storage scheme where the total capacity equals the sum of disk capacities does not provide any redundancy. Essentially, it is the same as RAID0. However, mergerfs still has a slight advantage over RAID0. When using mergerfs, all underlying file systems and disks are independent ext4 systems, so it is easy to expand the storage capacity. Additionally, if any disk fails and goes offline, it will not result in the loss of all data. As long as there is a proper backup, it only requires the recovery of data from the damaged disk. The cost is not too high, only a slight loss in performance. For personal file storage purposes, there is not much demand for performance, especially considering it is only 1GbE at home.

The problem lies in the aspect of proper backup. The purpose of choosing mergerfs instead of RAID is to reduce costs. Setting up a 4U rack with a bunch of HDDs at home is too expensive and noisy. Moreover, RAID provides redundancy, not backup. Just in case of accidental deletion or falling victim to ransomware within the local network, the data would still be at risk. Building offsite backup on your own also does not significantly reduce costs, as it still requires several high-speed spinning disks in one place. So, in the end, the focus turns to similar object storage solutions provided by cloud service providers.

After some research, I found that Restic perfectly meets my requirements:

It supports various data storage solutions, including one’s personal local filesystem or object storage like S3.
It provides a snapshot-based backup mode, allowing me to view the historical versions of multiple backups of the same directory using snapshots.
It supports incremental backups, only uploading the differences between multiple backups of the same directory.
It supports encryption, ensuring that the cloud service provider hosting the data is unaware of what I have uploaded.

After solving the issue of finding the tool, the next step is to figure out where to store it. Initially, I found that restic can support Century Internet’s OneDrive for Business through rclone. So, I reluctantly subscribed to Office E3 for a year, considering that the cost of 1500CNY/year is still cheaper than buying object storage from Alibaba Cloud or Tencent Cloud separately (I have about 1TB of data to backup, and using S3 would cost $35USD per month). Using various cloud storage options would significantly reduce the cost, and the reason for not using Baidu Cloud is simple - I just dislike Baidu. The remaining services like OneDrive/Google Drive are not usable in China. Later, Alibaba Cloud joined the cloud storage market and provided a product called Aliyun Drive, where you can purchase 8TB of storage space for approximately 168CNY. Additionally, it is very convenient to find tools to bridge Aliyun Drive into WebDAV, such as the one I am using: GitHub: aliyundrive-webdav. Moreover, rclone, which is supported by restic, also supports the WebDAV backend, so the whole process is now complete and satisfactory.

Since all the dependency issues are resolved, the remaining task is to deploy this entire package to the NAS. The process is not complicated. After deploying and starting aliyundrive-webdav, use “rclone config” to add a new remote. Finally, initialize the repository with “restic init -r rclone:xxx:xxx”. With the help of systemd timer and your own script, you can achieve scheduled incremental backups. Of course, the initial upload is inevitably slow, but it will improve significantly in subsequent backups.

Here is the first uploaded vnstat record, which shows that 1.31TB was uploaded in three days, very smoothly:

ntzyz@nas-v2 ~ % vnstat -h

 ens18  /  hourly

         hour        rx      |     tx      |    total    |   avg. rate
     ------------------------+-------------+-------------+---------------
     2023-02-14
         15:00    606.69 MiB |   34.45 GiB |   35.04 GiB |   83.62 Mbit/s
         16:00    610.81 MiB |   34.25 GiB |   34.85 GiB |   83.16 Mbit/s
         17:00    623.61 MiB |   33.99 GiB |   34.60 GiB |   82.56 Mbit/s
         18:00    566.85 MiB |   29.15 GiB |   29.70 GiB |   70.87 Mbit/s
         19:00      7.76 GiB |   34.40 GiB |   42.16 GiB |  100.60 Mbit/s
         20:00     58.64 GiB |   35.24 GiB |   93.88 GiB |  224.01 Mbit/s
         21:00     43.52 GiB |   35.01 GiB |   78.52 GiB |  187.36 Mbit/s
         22:00     38.33 GiB |   35.04 GiB |   73.37 GiB |  175.07 Mbit/s
         23:00    576.09 MiB |   27.50 GiB |   28.06 GiB |   66.96 Mbit/s
     2023-02-15
         00:00    560.47 MiB |   26.72 GiB |   27.27 GiB |   65.06 Mbit/s
         01:00    652.31 MiB |   31.49 GiB |   32.13 GiB |   76.66 Mbit/s
         02:00    646.58 MiB |   32.78 GiB |   33.41 GiB |   79.73 Mbit/s
         03:00    628.83 MiB |   31.41 GiB |   32.02 GiB |   76.41 Mbit/s
         04:00    670.06 MiB |   33.76 GiB |   34.42 GiB |   82.12 Mbit/s
         05:00    675.27 MiB |   34.27 GiB |   34.93 GiB |   83.34 Mbit/s
         06:00    677.09 MiB |   34.08 GiB |   34.74 GiB |   82.90 Mbit/s
         07:00    699.17 MiB |   33.83 GiB |   34.52 GiB |   82.36 Mbit/s
         08:00    579.34 MiB |   26.45 GiB |   27.02 GiB |   64.46 Mbit/s
         09:00    524.03 MiB |   23.60 GiB |   24.11 GiB |   57.54 Mbit/s
         10:00    600.53 MiB |   29.05 GiB |   29.64 GiB |   70.73 Mbit/s
         11:00    662.55 MiB |   32.28 GiB |   32.93 GiB |   78.58 Mbit/s
         12:00    695.57 MiB |   33.62 GiB |   34.30 GiB |   81.83 Mbit/s
         13:00    715.18 MiB |   33.52 GiB |   34.22 GiB |   81.64 Mbit/s
         14:00     58.99 MiB |    2.78 GiB |    2.84 GiB |   81.31 Mbit/s
     ------------------------+-------------+-------------+---------------

Site migration

Fri, 03 Feb 2023 00:00:00 +0000

Although there may not be many people reading this blog year-round, I still want to provide an explanation for such a big adjustment.

Why migration

The main adjustment is switching the blog framework from a self-made Node.js + Vue.js server application to Hugo. As for why the switch, the main reason is that I don’t want to continue maintaining a stateful service. Also, since the code is quite old and hasn’t been maintained, I really don’t want to add any new features to it anymore.

Currently, the content management engine for the site is Hugo, and I found a clean and minimalistic theme that supports dark mode. Keeping the content management static also has the advantage of easy deployment on Cloudflare Pages, enjoying the zero-origin access with high speed provided by AS13335’s global edge nodes.

What’s new

Previously my blog only provide Chinese posts, for now I’ll start to write post in English by default. I have already translate some of my older posts to English which I think is worth reading. If you are willing helping the translation please contact me :)

What has been changed

Since there are some private and non-public services running under other paths of ntzyz.io, it would be troublesome to directly host the blog on this domain: static resources still need to be deployed on a virtual machine located in Hong Kong, with Cloudflare…

Because there are some private and undisclosed services running on other paths of ntzyz.io, it would be troublesome to run a blog directly on this domain. Static resources still need to be deployed on virtual machines located in Hong Kong, and Cloudflare CDN still needs to frequently fetch data from the origin server, which is far less performant than a zero-origin-fetch setup like Cloudflare Pages. So, the final solution is to create a new domain, ntzyz.space, to host the blog content.

As for the previous static resources on ntzyz.io, such as the Grayscale Image Converter and CoreMark WebAssembly, they will continue to be served by ntzyz.io. In this case, the nginx configuration just needs to be handled simply.

error_page 404 = @redirect_to_space;
error_page 403 = @redirect_to_space;

location /wp-content {
  root /var/www/blog;
}

location / {
  root /var/www/ntzyz.io/static;
}

location @redirect_to_space {
  return 302 https://ntzyz.space$request_uri;
}

What about comments

Many times, comments don’t leave much useful information. If there are any suggestions or errors that need to be addressed, you can send an email to blog#ntzyz.io, or contact me on Telegram.

Search Telegram History with Elastic Stack

Thu, 03 Sep 2020 11:52:25 +0000

Telegram 的中文搜索是出了名的难用，只要涉及到中文、或者中英文混杂，就原地暴毙：

就比如遇到这种想找黑历史的时候，搜索功能摆在这里却用不了，只能原地感叹一句要你何用。忍忍总也有个限度，当我第 n 次想找历史记录但是无论如何也找不到的时候，一拍脑袋：

思路很简单嘛，公司里用 ELK 搜日志，聊天记录的本质就是聊天的日志，那干脆就用 ELK 来自建一个聊天记录搜索给自己用就好了嘛。这样来看，待实现的需求就只有历史数据的获取和聊天记录实时写入了。

部署一套 Elastic Stack 并不麻烦，最省事儿的途径就是直接用 docker。我这边呢继续用家里的 ESXi，开了一台独立的 VM，感觉没必要用 docker 再套一层，就稍微麻烦一点，从源里面安装 elasticsearch 和 kibana 就可以了，具体过程和配置都可以参照 Arch Wiki。

之后就是数据来源了。众所周知 Telegram 对二次开发是十分友好，所有人都可以直接用官方提供了 C++ 开发的 Telegram Database Library (TDLib) 来做一个具备完整功能的客户端。我这边具体要做的也就是获取一下会话列表，然后挑选其中几个把所有数据全都 POST 到 elastic 里就可以了。TDLib 提供了 JSON API，因此衍生出了各种语言的包装库，这里随便挑选一个顺手的，照着文档写也没什么难度。导入记录实际花销和代理质量以及水群多少有关系，我这边花了四五个小时吧。完成之后数据就可以在 Discover 中查找了：

当然利用 Kibana 提供的可视化功能，我们可以查看到很多很有趣的图表了，我随手创建了一些：

或者可以看看我关注的几个群/对话中消息在所有历史中的占比：

最后再回到开头的截图，咱们去 Elastic 里搜一下相同的关键字：

之后可以直接再去 Telegram 搜整句来定位上下文：

舒服了!

Load NVMe Driver in UEFI Shell

Wed, 27 Nov 2019 17:56:56 +0000

Months ago I bought an HP Z620 as a Hyper-V server at home. One day, I fond that my 500GB 860EVO SSD was almost full. Considering the factors of price, performance, and capacity, I decided to purchase a 2TB Intel DC P4500 SSD. However, after installing the SSD, I found that it was not recognized in the BIOS. After some research on Google, I discovered that although this workstation supports UEFI, it does not support NVMe SSDs. Therefore, I had to use some intermediate tools to boot the system from the NVMe SSD.

The simplest method was to use Clover Bootloader. Although this tool is primarily used for booting hackintosh, it can also be used as a general-purpose bootloader. It automatically searches for partitions and is quite user-friendly. However, when I was preparing to switch to ESXi for the server, I found that Clover couldn’t automatically detect the system. I didn’t want to manually write a custom entry (XML coding is really not user-friendly), so I decided to find another solution.

After some Google search, I found that UEFI itself supports loading drivers, so using UEFI Shell should solve the problem. First, I created an ESP partition on the GPT disk recognized by the BIOS. Then, I copied the UEFI Shell and NVMe drivers to the partition and created a startup.nsh file in the root directory, with the following content:

load -nc fs0:\NvmExpressDxe-64.efi
connect -r
map -u
fs1:\EFI\BOOT\BOOTX64.EFI

The content of the script is easy to understand, so I won’t explain it :)

Then, use the “efibootmgr” command, or use related tools in Windows PE, to create a boot entry with the executable file being the EFI Shell in that partition, and set it as the default. As for the partition and path aspect mentioned in the script, they may vary on each machine. You can directly enter the UEFI Shell, load the drivers, and then use the “map” command to check the path of the NVMe disk that needs to be bootable.

References

Collect Server Network Traffic with Multiple IPs on one same interface

Wed, 05 Jun 2019 09:52:43 +0000

假设某服务器有一个接口 eth0，并配置了三个 IP 地址 10.0.0.4, 10.0.0.5, 10.0.0.6。需求就是，如何知道这一台服务器每个 IP 的历史出网流量。

思路由 @a350063 提供，很简单，活用 iptables 的计数器就 OK 了。这种方法非常灵活，可以对任何能用 iptables 规则描述的流量数据进行统计，比如统计某个源地址指定端口的进出流量。首先我们使用 iptables 增加一些规则：

1
2
3

# iptables -I OUTPUT -s 10.0.0.4/32 -j ACCEPT
# iptables -I OUTPUT -s 10.0.0.5/32 -j ACCEPT
# iptables -I OUTPUT -s 10.0.0.6/32 -j ACCEPT

执行完成之后，就可以直接使用命令 iptables -nvxL OUTPUT 看到每个 IP 的出网流量了：

# iptables -nvxL OUTPUT
Chain OUTPUT (policy ACCEPT 3856 packets, 1914416 bytes)
    pkts      bytes target     prot opt in     out     source               destination
   12880   11794276 ACCEPT     all  --  *      *       10.0.0.4             0.0.0.0/0
    6850    5801012 ACCEPT     all  --  *      *       10.0.0.5             0.0.0.0/0
     110       9315 ACCEPT     all  --  *      *       10.0.0.6             0.0.0.0/0

接下来就是想办法记录这些数据了。由于我比较懒，不想用什么杂七杂八的东西来完成这个工作，直接用 shell 脚本。至于数据存到哪儿，肯定是数据库了，省事儿嘛。先建个表：

DROP DATABASE IF EXISTS traffic_collection;
CREATE DATABASE traffic_collection;
USE traffic_collection;

CREATE TABLE traffic_log (
  id INT PRIMARY KEY NOT NULL AUTO_INCREMENT,
  timestamp TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
  ip_address VARCHAR(40) NOT NULL,
  outgoing_bytes INTEGER NOT NULL
);

然后写个获取数据，插入数据库并重置计数器的脚本：

#!/bin/bash

set -e

RECORD_SQL="INSERT INTO traffic_log(ip_address, outgoing_bytes) VALUES "`iptables -nvxL OUTPUT | grep -P '10\.0\.0\.[4-6]' | awk '{ print "(\""$8"\","$2")," }' ORS='' | sed s/,$/\;/`
iptables -Z OUTPUT
mysql -u ****** --password=****** -D traffic_collection -e "$RECORD_SQL"

脚本内容很简单，构造 SQL，清空 iptables 计数器并执行 SQL。可能有人已经发现了这个脚本存在的问题：构造 SQL 和清空计数器之间的流量信息可能会丢失。比较好的方案应该是不清空计数器，而是将上一次的数据与本次的数据做差，将这个差值存入数据库。~~但是我懒，不想管了（~~

让这个脚本每小时执行一次，就完事儿了。创建文件 /etc/systemd/system/record_traffic.service：

[Unit]
Description=record traffic

[Service]
Type=oneshot
ExecStart=/bin/bash /root/record_traffic.sh

并创建 /etc/systemd/system/record_traffic.timer：

[Unit]
Description=record traffic hourly

[Timer]
OnCalendar=*-*-* *:00:00
Persistent=true

[Install]
WantedBy=timers.target

最后 enable && start 这个 timer 即可。运行了一段时间之后，数据库内就有了大概这样的数据：

MariaDB [traffic_collection]> select * from traffic_log limit 50 offset 2200;
+------+---------------------+------------+----------------+
| id   | timestamp           | ip_address | outgoing_bytes |
+------+---------------------+------------+----------------+
| 2201 | 2019-06-05 06:00:03 | 10.0.0.5   |        1649373 |
| 2202 | 2019-06-05 06:00:03 | 10.0.0.6   |          58858 |
| 2203 | 2019-06-05 07:00:01 | 10.0.0.4   |       16317328 |
| 2204 | 2019-06-05 07:00:01 | 10.0.0.5   |        2898066 |
| 2205 | 2019-06-05 07:00:01 | 10.0.0.6   |          45341 |
| 2206 | 2019-06-05 08:00:04 | 10.0.0.4   |       22077543 |
| 2207 | 2019-06-05 08:00:04 | 10.0.0.5   |        3361343 |
| 2208 | 2019-06-05 08:00:04 | 10.0.0.6   |          53434 |
| 2209 | 2019-06-05 09:00:04 | 10.0.0.4   |       34954806 |
| 2210 | 2019-06-05 09:00:04 | 10.0.0.5   |       22881420 |
| 2211 | 2019-06-05 09:00:04 | 10.0.0.6   |         179481 |
| 2212 | 2019-06-05 10:00:05 | 10.0.0.4   |       79252475 |
| 2213 | 2019-06-05 10:00:05 | 10.0.0.5   |       47689049 |
| 2214 | 2019-06-05 10:00:05 | 10.0.0.6   |         979906 |
+------+---------------------+------------+----------------+
14 rows in set (0.001 sec)

~~这个 timer 很不准啊~~

Linux Desktop on Dual Monitor with Different DPI

Mon, 15 Apr 2019 15:16:39 +0000

Very simple, just render everything at the highest DPI and then compress it for presentation on a low-resolution screen. With this approach, the low DPI screen will be “supersampled” and may be slightly blurry, but the loss is not significant compared to the approach of high-resolution screens presenting low-resolution images.

For example, here I have two 27-inch monitors, connected to DP1 and HDMI2 respectively (you can use xrandr to check). DP1 is a Dell 2K monitor (2560x1440), and HDMI 2 is a Lenovo FHD monitor (1920x1080). In terms of screen positioning, FHD is on the left and 2K is on the right.

First, you need to disable the window manager’s built-in multi-screen manager (such as KDE’s Kscreen2), if there is one. Then, in ~/.xinitrc or any equivalent X11 startup script, add the following commands:

xrandr \
  --output HDMI2 \
    --panning 2400x1350 \ # Left screen render resolution
    --pos 0x0 \           # Top left position of the left screen
    --scale 1.25x1.25 \   # Scaling factor of the left screen
  --output DP1 \
    --panning 2560x1440 \ # Right screen render resolution
    --pos 2560x0 \        # Top left position of the right screen
  --fb 4960x1440          # Size of the entire framebuffer

Some of the values here are my parameters. Different multi-monitor configurations may require a slight calculation on your own, which is not difficult.

After completion, restart X server to complete the adjustment. If all goes well, the two screens should work relatively normally. The effect is shown in the picture below:

About

Thu, 28 Feb 2019 00:00:00 +0000

Front-end developer @ ByteDance Inc.
Operator of Yuze Network (AS142641)
Living in Shanghai
Code on macOS/Arch Linux, but use Debian/NixOS on production
Sony alpha mirrorless camera user
Learning piano
E-mail: $(base64 -d <<< aUBudHp5ei5pbwo=)

Solution for Internet Connection Sharing Broken after Reboot on Windows 10

Sun, 09 Sep 2018 00:00:00 +0000

2020-02-25 更新：

今天在某 Telegram 群里摸鱼的时候又扯到了这个话题，群友提出了更科学的解决方法，直接修改注册表让 Windows 启用持续化的网络连接共享，具体来说就是创建一个注册表项：

Path: HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\SharedAccess
Type: DWORD
Setting: EnableRebootPersistConnection
Value: 1

值得注意的是，这个方法只可以在 Windows 10 1709 以上，且安装了 KB 4054517 更新的计算机上生效，具体可以查看微软的文档：ICS doesn’t work after computer or service restart on Windows 10

在上海租的房子只提供了无线接入（802.11n）的方式来访问互联网，而我个人又比较希望自己能获得一个更快的无线内网，并把自己的网络设备对外部隐藏，所以我就购入了一个 NETGEAR A6210 的无线网卡，支持 802.11AC，USB 3.0 界面，准备做一个这样的网络连接：

+--------------------+       +-----------------------+
| LAN(802.11AC + GbE)| CAT5E |  HP ProLiant MS Gen8  |
| (PS4, Kindle, etc) +-------+  eth0: 192.168.137.1  |
|  192.168.137.0/24  |       | wlan0: 192.168.0.x/24 |
+--------------------+       +---+-------------------+
                                 | 802.11n
                             +---+------------+
                             | TP-LINK Router |
                             +----------------+

这种结构对于 Linux PC 来说十分简单，只需要开启 ip_forward 并执行 iptables -t nat -A POSTROUTING -j MASQUERADE 就能做到，但是可惜这块 USB3.0 的无线网卡主线内核里并没有驱动，MTK 开源的驱动在 OpenWrt 下性能又非常糟糕，无奈只能选择使用 Windows 来做路由。

本来呢对于 Windows 来说配置这样的网络也很简单，右击无线网卡的适配器选择共享网络连接就好了，然而万恶的微软在网络共享的实现上埋了个 bug：重启电脑后网络共享会失效，表现为被共享的接口仍然能提供 DHCP 服务，但是 IP Forward 却完全不可用。进入网络连接，取消共享后再执行一次共享，问题就能解决。

那么我们只需要准备一个计划任务让系统每次开机都重复一遍这个操作就 OK 了，写一段 PowerShell 脚本即可：

$NetShare = New-Object -ComObject HNetCfg.HNetShare
$wlan = $null
$ethernet = $null

foreach ($int in $NetShare.EnumEveryConnection) {
  $props = $NetShare.NetConnectionProps.Invoke($int)
  if ($props.Name -eq "WLAN") {
    $wlan = $int;
  }
  if ($props.Name -eq "网桥") {
    $ethernet = $int;
  }
}

$wlanConfig = $NetShare.INetSharingConfigurationForINetConnection.Invoke($wlan);
$ethernetConfig = $NetShare.INetSharingConfigurationForINetConnection.Invoke($ethernet);

$wlanConfig.DisableSharing();
$ethernetConfig.DisableSharing();

$wlanConfig.EnableSharing(0);
$ethernetConfig.EnableSharing(1);

参考资料

NetConnectionSharing (PowerShell Module)

Migrate to Walkman From NetEase Cloud Music

Mon, 09 Jul 2018 00:00:00 +0000

今年六月份突然脑抽想买个很“传统”的 MP3 回来玩玩，于是就从同学那收了这么一个大法的播放器：NW-A45。到手之后检查了一下机器，几乎是全新成色，带一个 64GB 的 TF 卡，非常兴奋。但当我接上了电脑准备开始快乐地复制音乐时，我突然意识到一个问题：我从哪里复制呢？

在高中的时候，我还是保持着从网上收集音乐并整理好的习惯，原因是当时手里的播放器是 Apple iPod nano 6，整理妥当的音乐在使用 iTunes 同步时会方便很多很多。不过后来被人安利了网易云音乐之后，这个收集和整理的习惯就彻底停下来了，原因同样很简单，在线的音乐服务使用起来比自己整理要便捷太多太多。也因为如此，iPod nano 正式退休，开始养（吃）老（灰）。这之后我都是一直使用着网易云音乐，为了解决手机 16GB 存不下音乐的问题还购买了一个 iPod Touch，64G 的版本拿来听歌显然时毫无问题的。就这样，网易云音乐 + iPod Touch 的组合用到了现在。

回到从前的整理音乐+使用工具同步肯定是不会去做的，这辈子都不可能。然而 Walkman 买都买了，不把这个设备用起来就很不服气。仔细思考了一下，网易云音乐上的歌曲其实是可以随意下载的（会员才能下无损，但这不是重点），不过下载了歌曲是会丢失当时整理歌单时的顺序和分类，这一点说重要也不怎么重要，但是我很不愿意半夜听歌时随机到一些奇怪的音乐（比如血源诅咒的 OST，怕不是半夜直接吓醒），所以歌单这一信息还是有必要保留下来的。

说到保留歌单，第一个想到的办法是，准备好一组文件夹，每个文件夹存放对应歌单的内容。然后逐个将网易云音乐的歌单下载，每下载完一个歌单之后，将下载目录的歌全部复制到对应的歌单中。这个方法刚开始觉得似乎还能用，但是稍微想一下就不难发现，该方法会导致文件系统里有很多的冗余数据（比如某首歌同时存在于若干个歌单中），同时也无法很好的增量下载新歌曲。

那么换个思路，网易云音乐是能够离线播放音乐的，歌单数据离线时也是可以访问的，那么就能说明至少他在本地会保存一些数据文件来存储歌单啊什么的信息。这样我只要想办法找到他们，从中解析出我所需要的信息即可。对着 AppData 目录找了一下果然不出意外，在 Local\Netease\CloudMusic\Library 内找到了一些看起来非常像那么一回事儿的文件：

PS C:\Users\ntzyz\AppData\Local\Netease\CloudMusic\Library> Get-ChildItem .


    目录: C:\Users\ntzyz\AppData\Local\Netease\CloudMusic\Library


Mode                LastWriteTime         Length Name
----                -------------         ------ ----
-a----         2018/7/8     23:24       13161472 library.dat
-a----         2018/7/8     23:24          16384 library.dat-journal
-a----         2018/7/7     12:40       13161472 library_bak.dat
-a----         2018/7/9     14:12       15628288 webdb.dat

打开 Cygwin 64 对着 webdb.dat 执行一发 file 命令，结果是令人意外的惊喜（赞美 libmagic）：

1
2

% file webdb.dat
webdb.dat: SQLite 3.x database, last written using SQLite version 3007006

那可就好办了啊，直接打开 SQLiteDatabaseBrowser 看着表结构，然后对着 Visual Studio 2017 一顿输出，所需要的导出功能就能很轻松地实现了。代码放在最后，稍微有点长。

解决完导出问题后，剩下的问题就是，如何增量同步数据和解决网易云音乐新出现的 .ncm 版权保护文件了。增量同步本身没有难度，毕竟现成的工具非常多，听说 Windows 的 RoboCopy 可以完成，隔壁 Rsync 也不是不能做，所以这个问题最后解决。

而 .ncm 这一问题就稍稍蛋疼了，从 SQLite 数据库中不难看出，他其实是常用媒体格式的一个封装（数据库中有 real_suffix 这么个字段），但是按照网易的尿性，这个封装肯定是对整个文件进行了加密。不过既然能离线播放，那么必然是将解密相关的密钥啥的保存在了客户端里，而这个客户端又是一个 Qt Webkit 套壳的 Web 应用程序，用脚想也觉得网易肯定把密钥保存到了 Qt 的那一部分里了，感觉上反编译不可避。逆向工程这个方面我是一窍不通，于是就……尝试在 Google 上寻求帮助（丢人！），发现了一个神秘仓库，实现了对这一格式的解封装。这里按照仓库 README 的要求就不贴链接了，但是简单的提一下那段代码里有的 bug。

OpenSSL 创建 cipher 的接口稍有变更，改成这个样式就行了：
1

EVP_CIPHER_CTX* x = EVP_CIPHER_CTX_new();
文件操作，这个哥们用的是 ISO C 提供的文件 IO 方式，在打开的时候描述打开方式的字符串使用了 "r"，按照标准就是文本文件而不是二进制文件，在 GNU C Library 上使用 fread 读取一个 uint32_t 后文件流只会向后移动四个字节，但是在 mingw 编译出来的程序表现则是一直读取到了换行符，文件流位置挪了一大截，后续操作也就全都爆炸了。@ZephRay 表明标准库并没有明确规定文件在使用 "r" 模式打开时，使用 fread 函数读取数据后文件流的变化，也就是吃了个 UB 的亏。修起来也是简单，全都加个 b 就行了。

解决完这个 dump 之后，就需要解决增量同步的问题了。我的预期同步逻辑大概是这样的：

同步数据
解密并删除 .ncm 文件
生成 m3u8 播放列表文件

问题就在第二部的解密并删除了。如果删除 ncm 文件，那么下一次同步又会把他们全都拷过来，如果不删这个空间又浪费了很多（英文歌很多都是这个格式）。想来想去还是干脆自己写一段脚本来完成吧，没啥难度的事。

$srcDir = "D:\CloudMusic\*";
$destDir = "F:\MUSIC\CloudMusic";

$fileList = Get-Item -Path $srcDir;

foreach ($fileItem in $fileList) {
    $destPath = $($destDir + "\" + $fileItem.Name);

    if (Test-Path -LiteralPath $destPath) {
        $destFile = Get-Item -LiteralPath $($destDir + "\" + $fileItem.Name);

        if ($destFile.Length -eq $fileItem.Length) {
            continue;
        }

        Write-Output $("[WARN]: File Size Mismatch: " + $fileItem.Name);
        continue;
    } else {
        if ($fileItem.Name -match '\.ncm$') {
            $fileNameWithoutSuffix = $fileItem.Name -replace '.ncm';
            # Write-Output $("[INFO]: NCM File Found: " + $fileNameWithoutSuffix);
            $matchedFileList = Get-Item -Path $($destDir + "\" + $fileNameWithoutSuffix + ".*");
            if ($matchedFileList.Length -eq 0) {
                Write-Output $("[INFO]: File Not Found: " + $fileItem.Name + ", Copying...");
                Copy-Item $fileItem -Destination $destDir
            }
        } else {
            Write-Output $("[INFO]: File Not Found: " + $fileItem.Name + ", Copying...");
            Copy-Item $fileItem -Destination $destDir
        }
    }
}

这份代码逻辑很简单，如果文件不存在，同时相同文件名、不同后缀名的文件也不存在的话，就去复制这个文件，其他情况就跳过。需要注意的是这个脚本不提供递归复制的支持和文件一致性的检验，毕竟没打算实现一个全面的同步工具（

有了这些东西就能很顺利的使用 Walkman 听歌了呢。折腾折腾其实也挺有趣的（（

导出 M3U8 代码：

using Newtonsoft.Json;
using Newtonsoft.Json.Linq;
using System;
using System.Collections.Generic;
using System.Data.SQLite;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;

namespace ncm_playlist_export
{
    class PlayList
    {
        public Int64 pid;
        public dynamic playlist;
    }

    class SongDetail
    {
        public dynamic detail;
        public string relative_path;
        public string real_suffix;
    }

    class FailedRecorods
    {
        public Int64 tid;
        public string reason;

        public FailedRecorods(Int64 _in_tid, string _in_reason)
        {
            tid = _in_tid;
            reason = _in_reason;
        }
    }

    class Program
    {
        static SQLiteConnection conn = null;

        static List<PlayList> FetchPlaylist()
        {
            var ret = new List<PlayList>();

            var cmd = conn.CreateCommand();
            cmd.CommandText = "SELECT * FROM web_playlist";

            var reader = cmd.ExecuteReader();

            while (reader.Read())
            {
                var elem = new PlayList();
                elem.pid = Convert.ToInt64(reader["pid"]);
                elem.playlist = JObject.Parse(reader["playlist"].ToString());

                ret.Add(elem);
            }

            return ret;
        }

        static List<Int64> FetchPlaylistSongs(Int64 pid)
        {
            var ret = new List<Int64>();

            var cmd = conn.CreateCommand();
            cmd.CommandText = "SELECT pid, tid FROM web_playlist_track WHERE pid = " + pid + " ORDER BY `order`";

            var reader = cmd.ExecuteReader();

            while (reader.Read())
            {
                ret.Add(Convert.ToInt64(reader["tid"]));
            }

            return ret;
        }

        static SongDetail FetchSongDetail (Int64 tid)
        {
            var ret = new SongDetail();
            var cmd = conn.CreateCommand();
            cmd.CommandText = "SELECT detail, relative_path, real_suffix FROM web_offline_track WHERE track_id =" + tid;

            var reader = cmd.ExecuteReader();

            if (reader.Read())
            {
                ret.detail = JObject.Parse(reader["detail"].ToString());
                ret.relative_path = reader["relative_path"].ToString();
                ret.real_suffix = reader["real_suffix"].ToString();

                return ret;
            }

            return null;
        }

        static string GetSafeFilename(string filename)
        {
            return string.Join("_", filename.Split(Path.GetInvalidFileNameChars()));
        }

        static void Main(string[] args)
        {
            var localAppData = Environment.GetEnvironmentVariable("LocalAppData");
            v arlrcPath = Environment.GetEnvironmentVariable("LocalAppData") + "\\Netease\\CloudMusic\\webdata\\lyric";
            conn = new SQLiteConnection("Data Source=" + localAppData + "\\Netease\\CloudMusic\\Library\\webdb.dat;Version=3;New=True;Compress=True;");
            conn.Open();

            var playlists = FetchPlaylist();
            int n = 1;

            foreach (var playlist in playlists)
            {
                int songsCount = 0, skippedCount = 0;
                string m3u8 = "";
                var songs = FetchPlaylistSongs(playlist.pid);
                var skipeedReasons = new List<FailedRecorods>();

                if (songs.Count == 0)
                {
                    continue;
                }

                for (var i = 0; i < songs.Count; i++)
                {
                    var detail = FetchSongDetail(songs[i]);

                    if (detail == null)
                    {
                        skipeedReasons.Add(new FailedRecorods(songs[i], "Detail data not found."));
                        skippedCount++;
                        continue;
                    }

                    if (detail.relative_path == "")
                    {
                        skipeedReasons.Add(new FailedRecorods(songs[i], "Relative path is empty."));
                        skippedCount++;
                        continue;
                    }

                    if (detail.relative_path.Substring(detail.relative_path.Length - 3) == "ncm")
                    {
                        detail.relative_path = detail.relative_path.Replace("ncm", detail.real_suffix);
                    }

                    if (false && File.Exists(lrcPath + "\\" + songs[i]))
                    {
                        var lrcJson = File.ReadAllText(lrcPath + "\\" + songs[i]);
                        dynamic lrc = JObject.Parse(lrcJson);

                        if (lrc["nolyric"] != true)
                        {
                            File.WriteAllText(Path.ChangeExtension(detail.relative_path, "lrc"), (string)lrc.lrc.lyric);
                        }
                    }

                    songsCount++;
                    m3u8 += "#EXTINF:" + detail.detail.duration + "," + detail.detail.name + "\n";
                    m3u8 += "Cloudmusic/" + detail.relative_path.Replace('\\', '/') + "\n\n";
                }

                if (songsCount == 0)
                {
                    continue;
                }

                System.IO.File.WriteAllText(n.ToString("00") + ". " + GetSafeFilename("" + playlist.playlist.name) + ".m3u8", m3u8);
                Console.WriteLine("Finish generating playlist: " + playlist.playlist.name + ", Songs count: " + songsCount + ", Skipped count: " + skippedCount);
                if (skipeedReasons.Count != 0)
                {
                    for (int i = 0; i < (skippedCount < 5 ? skippedCount : 5); i++)
                    {
                        Console.WriteLine("Track ID: " + skipeedReasons[i].tid + ", Reason: " + skipeedReasons[i].reason);
                    }
                }
                n++;
                Console.WriteLine("");
            }

            Console.WriteLine("Finished!");
        }
    }
}

Decode and playback MP3 in web browser using Web Assembly

Mon, 26 Feb 2018 00:00:00 +0000

~~纯属闲得蛋疼~~

实际运行效果

首先我们需要准备好 WebAssembly 的工具链（大概可以这么叫吧），此处请参考 Compiling from C/C++ to WebAssembly | MDN 中的步骤来完成。对于 Arch Linux 用户，可以从 AUR 中安装 emsdk：

`1`	`yaourt -S emsdk`

之后则是和 MDN 中的步骤一样，编译并配置 LLVM 和 Emscripten SDK：

1
2

sudo emsdk install --build=Release sdk-incoming-64bit binaryen-master-64bit
sudo emsdk activate --global --build=Release sdk-incoming-64bit binaryen-master-64bit

完成后，将相关目录添加到环境变量，方法很多，例如在 .zshrc 最后加上：

export PATH=$PATH:/usr/lib/emsdk:/usr/lib/emsdk/clang/fastcomp/build_incoming_64/bin:/usr/lib/emsdk/node/8.9.1_64bit/bin:/usr/lib/emsdk/emscripten/incoming:/usr/lib/emsdk/binaryen/master_64bit_binaryen/bin

然后就是编译一下 libmad，方法也不难，首先前往 SourceForge 下载 libmad 的源码，然后进入所在目录并解压，执行 configure, make, make install 三连：

cd /path/to/libmad.tar.gz/
tar xzf libmad*.tar.gz
cd libmad*
mkdir ../build
/configure --prefix=$(pwd)/../build CC=emcc
make
make install

这样我们就将 libmad 编译安装到了 ../build 目录下了。之后就是编写 C 的程序来利用 libmad 进行解码，这里我使用的是 libmad 提供的 High-level API。在放代码之前，需要简单的讲一下 libmad 是如何工作的。

libmad 的高层 API 提供了 struct mad_decoder 类型，在初始化 struct mad_decoder 时我们需要向初始化函数传递若干函数指针作为参数。在解码过程中，libmad 获得待解码数据、输出解码后的数据、抛出错误等行为都是通过这些函数指针来完成的。libmad 在适当的时机会调用我们提供的回调，这样就能用最简单的方式实现流试的 MP3 解码了。

以输入函数 enum mad_flow (*input_func)(void *, struct mad_stream *) 为例，它是 mad_decoder_init 函数的第二个参数，当 libmad 需要获得待解码的 MP3 数据时，这个函数会被调用，此时我们应该通过 mad_stream_buffer 函数向 input_func 的第二个参数 struct mad_stream * 中写入数据的首地址和长度。libmad 会根据这个函数的返回值判断是否存在后续数据，即如果我返回 MAD_FLOW_STOP，libmad 就不会再次调用 input_func，而是等待缓存的数据解码完成后退出。

每一个回掉函数的首个参数都是一个 void * 的指针，其中的内容时我们初始化 decoder 时提供的一个地址，用作当前解码的上下文变量。

以下晒代码：

#include <mad.h>
#include <stdint.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#define X86_64

#ifndef X86_64
#include <emscripten.h>
#endif

// 上文所述的上下文变量的结构体，用来保存解码的 PCM 数据和其他信息
struct buffer {
  uint8_t* input;
  size_t input_length;
  mad_fixed_t** output;
  size_t output_length;
  size_t output_channels;
  size_t output_samples;
};

// native 环境下 decode_mp3_to_pcm 函数的返回值
// 主函数通过读取这些信息将 PCM 写入存储
// wasm 环境下没用到这个结构体
struct output_t {
  mad_fixed_t** p_output;
  size_t channels;
  size_t samples;
};

// 输入回调
static enum mad_flow input_callback (void *data, struct mad_stream *stream) {
  struct buffer *buffer = (struct buffer *)data;

  // 已经没有数据了，直接不管了，返回结束
  if (buffer->input_length == 0) {
    return MAD_FLOW_STOP;
  }

  // 直接把整个 MP3 丢进去……反正又不是不能用（
  mad_stream_buffer(stream, buffer->input, buffer->input_length);
  buffer->input_length = 0;
  return MAD_FLOW_CONTINUE;
}

// 输出回调
static enum mad_flow output_callback (void *data, struct mad_header const *header, struct mad_pcm *pcm) {
  struct buffer *buffer = (struct buffer *)data;
  uint16_t samples;
  int i;

  samples = pcm->length;

  if (!buffer->output) {
    // 首次输出，分配好内存啥的
    puts("output func fired");
    buffer->output = (mad_fixed_t **)malloc(pcm->channels * sizeof(mad_fixed_t *));
    buffer->output_channels = pcm->channels;

    for (i = 0; i != pcm->channels; ++i) {
      // 到后面再 realloc，这个不算未定义行为吧
      buffer->output[i] = (mad_fixed_t *) malloc(0);
    }
  }

  #ifdef X86_64
  // 对于 native 环境这里用 realloc 就行了，虽然很粗暴，不过性能也不会差到哪儿去（丢人！）
  for (i = 0; i != pcm->channels; ++i) {
    buffer->output[i] = (mad_fixed_t *) realloc(buffer->output[i], (buffer->output_samples + pcm->length) * sizeof(mad_fixed_t));
  }

  // 逐个采样写入到输出数组
  for (i = 0; i != pcm->length; ++i) {
    int channel = 0;
    for (channel = 0; channel != pcm->channels; ++channel) {
      buffer->output[channel][buffer->output_samples + i] = pcm->samples[channel][i];
    }
  }

  #else
  // 至于 WASM ……是真的慢，不如直接把数据通过 EM_ASM_ 直接写入给 JavaScript 上下文，速度还是很快的……
  EM_ASM_({
    const begin = new Date().getTime();
    const pcm_l_addr = $0;
    const pcm_r_addr = $1;
    const frames = $2;
    const offset = $3;

    if (!window.pcm) {
      // 这里如果直接用字面值构造变量，emcc 会和我说 left 和 right 这两个变量为定义
      // 就很神秘，不过显然走 JSON.parse 就没这个问题啦
      window.pcm = JSON.parse("{ \"left\": [], \"right\": [] }");
      window.begin = new Date().getTime();
      window.jscost = 0;
    }
    let i = 0;
    for (i = 0; i < frames; ++i) {
      // 这里的 HEAP32 可以理解为一个 int32_t 的数组，里面就是 WASM 的堆内存
      // 我们从 EM_ASM_ 宏的参数中传入了两个 PCM 采样数组的地址，计算偏移就能拿到数据了
      // 我们直接把数据写入 JS 数组，反正众所周知 JS 的数组是稀疏数组，也不需要人为的扩展大小
      window.pcm.left[offset + i] = Module.HEAP32[pcm_l_addr / 4 + i];
      window.pcm.right[offset + i] = Module.HEAP32[pcm_r_addr / 4 + i];
    }

    window.jscost += (new Date().getTime()) - begin;
  }, pcm->samples[0], pcm->samples[1], pcm->length, buffer->output_samples);
  #endif

  buffer->output_samples += pcm->length;

  // 当然是继续啦，就算没有数据了这个函数也看不出来，反正没数据了 libmad 也不会再去执行这个回调了
  return MAD_FLOW_CONTINUE;
}

// 异常回调
static enum mad_flow error_callback(void *data, struct mad_stream *stream, struct mad_frame *frame) {
  puts("error func fired");
  // doing nothing（
  return MAD_FLOW_CONTINUE;
}

// 解码函数，在这里构造 libmad 的相关数据并执行解码，目的是可以直接被 wasm 拿去用
#ifndef X86_64
EMSCRIPTEN_KEEPALIVE void
#else
struct output_t
#endif
decode_mp3_to_pcm(uint8_t* input, size_t input_size) {
  struct buffer buffer;
  struct mad_decoder decoder;
  struct output_t output;
  int r;

  memset(&buffer, 0, sizeof(buffer));
  buffer.input = input;
  buffer.input_length = input_size;

  mad_decoder_init(&decoder, &buffer, input_callback, NULL, NULL, output_callback, error_callback, NULL);
  mad_decoder_options(&decoder, 0);
  r = mad_decoder_run(&decoder, MAD_DECODER_MODE_SYNC);
  mad_decoder_finish(&decoder);

  output.p_output = buffer.output;
  output.samples = buffer.output_samples;
  output.channels = buffer.output_channels;
  #ifdef X86_64
  return output;
  #else
  EM_ASM_({
    console.log('WASM part finished, duration: ' + ((new Date().getTime()) - window.begin - window.jscost) + 'ms');
    const channels = $0;
    const samples = $1;
    const pcm_l_addr = $2;
    const pcm_r_addr = $3;

    let ctx = new AudioContext(); // 创建音频上下文
    let frames = samples;
    let audioBuffer = ctx.createBuffer(channels, frames, 44100); // 创建音频缓冲区
    let left = audioBuffer.getChannelData(0); // 获得两个声道的 PCM 数据数组
    let right = audioBuffer.getChannelData(1);
    let i = 0;

    // 往两个数组里写入数据，值得注意的是这里每个采样的取值范围是 [-1. 1]，
    // 如果直接把 libmad 的输出 (mad_fixed_t aka signed long）写进去
    // 回放的效果大概就是原歌曲音量放大 2 ^ 30 并强行限幅后的方波音乐了吧（
    // 反正第一次我忘了这事儿，回放出来的东西把我吓死了（
    for (i = 0; i < frames; ++i) {
      left[i] = window.pcm.left[i] / (2 ** 30);
      right[i] = window.pcm.right[i] / (2 ** 30);
    }

    // 将音频缓冲变为音频源的数据源，然后输出到音频上下文
    let source = ctx.createBufferSource();
    source.buffer = audioBuffer;
    source.connect(ctx.destination);
    // 开始播放咯
    source.start();
  }, output.channels, output.samples, output.p_output[0], output.p_output[1]);
  #endif
}

#ifndef X86_64
EMSCRIPTEN_KEEPALIVE
#endif
// 当时拿来测试能不能成功执行一个函数的，懒得删了
void hello () {
  printf("hello world\n");
}

#ifndef X86_64
EMSCRIPTEN_KEEPALIVE
#endif
// 测试 typed array 和 wasm 数组的转换
// 懒得删了
void test(uint8_t *array, size_t length) {
  int i = 0;
  for (i = 0; i != length; ++i) {
    printf("%d ", array[i]);
  }
  printf("\n");
}

#ifndef X86_64
EMSCRIPTEN_KEEPALIVE
#endif
int main() {
  #ifdef X86_64
  // native 环境直接打开文件转换然后写入到外部的一个文件
  // 就行了，主要是测试解码的代码能不能跑通的
  FILE* fp = fopen("audio.mp3", "rb");
  size_t file_size = 0;
  uint8_t *input;
  int i;

  if (!fp) {
    printf("Fuck you\n");
    return 1;
  }

  fseek(fp, 0, SEEK_END);
  file_size = ftell(fp);

  input = (uint8_t *) malloc(file_size);
  fseek(fp, 0, SEEK_SET);
  fread(input, file_size, 1, fp);
  fclose(fp); fp = NULL;

  struct output_t output = decode_mp3_to_pcm(input, file_size);

  fp = fopen("output.pcm", "wb+");
  for (i = 0; i != output.samples; ++i) {
    int channel = 0;
    for (channel = 0; channel != output.channels; ++channel) {
      fwrite(&output.p_output[channel][i], sizeof(output.p_output[channel][i]), 1, fp);
    }
  }
  fclose(fp);
  #else
  // WASM 环境不需要执行 main，而是通过 JS 直接调用某一个 export 出来的函数
  // 随便跑点东西搞点输出，意思一下就行了（
  hello();
  #endif
  return 0;
}

比较重要的 JS 相关的代码都在注释里说明作用了。然后只要编译一下这个代码就行啦！这里我给了比较多的内存，主要是防止他崩掉（

1
2

mkdir server
emcc -I../build/include -L../build/lib main.c -lmad -s WASM=1 -o server/test.html -s TOTAL_MEMORY=268435456 -s EXTRA_EXPORTED_RUNTIME_METHODS='["ccall","cwrap","writeArrayToMemory"]' -O3

顺利编译后，随便找一个 MP3 的音频，放到 server 文件夹中，并启动一个 HTTP 服务器，root 就是这个 server 文件夹，比如

const express = require('express');
const server = express();
server.use('/', express.static('./server');
server.listen(8080);

不出意外的话，打开浏览器，访问 localhost:8080/test.html 就能看到一个看起来像终端的玩意儿里面输出了一行 Hello World 了。

现在我们再打开 JavaScript 控制台，调用 decode_mp3_to_pcm 函数来播放音频：

(async function () {
  mp3 = await fetch('/audio.mp3'); // 换成你自己的音频 URI
  data = new Uint8Array(await mp3.arrayBuffer());

  buffer = Module._malloc(data.length);
  Module.writeArrayToMemory(data, buffer);

  Module.ccall('decode_mp3_to_pcm'
              'null',
              ['number', 'number'],
              [buffer, data.length])
})();

就成功啦，libmad 在我的笔记本上（i5 4200H，Google Chrome 64）需要花约 1.166 秒来进行解码，性能还算不错了吧……

Create Linux Container with systemd-nspawn

Thu, 21 Dec 2017 00:00:00 +0000

使用 systemd-nspawn 这个命令我们可以很方便的创建一个 Linux 容器，需要的只是一个使用 systemd 作为 init 的 Linux 发行版的根文件系统。通过创建容器，我们可以获得一个可以随便折腾而不用担心损坏的 Linux 环境。这里用 Ubuntu 16.04 和 CentOS 7 为例，整个过程可以说是非常简单（虽然比起 Docker 还是麻烦了点）

对于 Ubuntu，可以直接从源里下载到它的根文件系统。下载一份，并解压到 /var/lib/machines/ubuntu1604：

1
2
3

sudo mkdir -p /var/lib/machines/ubuntu1604
wget http://mirrors.ustc.edu.cn/ubuntu-cdimage/ubuntu-base/releases/16.04.3/release/ubuntu-base-16.04.1-base-amd64.tar.gz -O /tmp/rootfs.tgz
sudo tar xpzf /tmp/rootfs.tgz -C /var/lib/machines/ubuntu1604

OK，到此为止我们就得到了一个可以被 systemd-nspawn 启动的 rootfs，不过我们还需要一些配置，例如修改 root 密码等等：

1
2

chroot /var/lib/machines/ubuntu1604 /usr/bin/passwd root
echo ubuntu > /var/lib/machines/ubuntu1604/etc/hostname

下面只需要用 systemd-nspawn 来“启动”这个容器：

`1`	`systemd-nspawn -b -D /var/lib/machines/ubuntu1604 --bind=/lib/firmware`

这样就完成了！相当简单吧～输出内容大概是这样：

Spawning container ubuntu1604 on /var/lib/machines/ubuntu1604.
Press ^] three times within 1s to kill container.
systemd 229 running in system mode. (+PAM +AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ -LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD -IDN)
Detected virtualization systemd-nspawn.
Detected architecture x86-64.

Welcome to Ubuntu 16.04.1 LTS!

Set hostname to <ubuntu>.
Failed to install release agent, ignoring: No such file or directory
[  OK  ] Listening on Journal Socket (/dev/log).
[  OK  ] Started Dispatch Password Requests to Console Directory Watch.
[  OK  ] Started Forward Password Requests to Wall Directory Watch.
[  OK  ] Reached target Paths.
[  OK  ] Reached target Remote File Systems (Pre).
[  OK  ] Reached target Remote File Systems.
[  OK  ] Listening on /dev/initctl Compatibility Named Pipe.
[  OK  ] Created slice System Slice.
[  OK  ] Reached target Slices.
[  OK  ] Created slice system-getty.slice.
[  OK  ] Reached target Swap.
[  OK  ] Reached target Encrypted Volumes.
[  OK  ] Listening on Journal Socket.
         Mounting Huge Pages File System...
[  OK  ] Reached target Sockets.
         Starting Remount Root and Kernel File Systems...
         Mounting POSIX Message Queue File System...
         Starting Journal Service...
         Mounting FUSE Control File System...
[  OK  ] Mounted POSIX Message Queue File System.
[  OK  ] Mounted Huge Pages File System.
[  OK  ] Mounted FUSE Control File System.
[  OK  ] Started Remount Root and Kernel File Systems.
[  OK  ] Reached target Local File Systems (Pre).
[  OK  ] Reached target Local File Systems.
         Starting Load/Save Random Seed...
[  OK  ] Started Load/Save Random Seed.
[  OK  ] Started Journal Service.
         Starting Flush Journal to Persistent Storage...
[  OK  ] Started Flush Journal to Persistent Storage.
         Starting Create Volatile Files and Directories...
[  OK  ] Started Create Volatile Files and Directories.
         Starting Update UTMP about System Boot/Shutdown...
[  OK  ] Reached target System Time Synchronized.
[  OK  ] Started Update UTMP about System Boot/Shutdown.
[  OK  ] Reached target System Initialization.
[  OK  ] Started Daily Cleanup of Temporary Directories.
[  OK  ] Reached target Basic System.
         Starting Permit User Sessions...
         Starting LSB: Set the CPU Frequency Scaling governor to "ondemand"...
         Starting /etc/rc.local Compatibility...
[  OK  ] Started Daily apt activities.
[  OK  ] Reached target Timers.
[  OK  ] Started Permit User Sessions.
[  OK  ] Started /etc/rc.local Compatibility.
[  OK  ] Started Console Getty.
[  OK  ] Reached target Login Prompts.
[  OK  ] Started LSB: Set the CPU Frequency Scaling governor to "ondemand".
[  OK  ] Reached target Multi-User System.
[  OK  ] Reached target Graphical Interface.
         Starting Update UTMP about System Runlevel Changes...
[  OK  ] Started Update UTMP about System Runlevel Changes.

Ubuntu 16.04.1 LTS ubuntu console

ubuntu login:

值得注意的是，这个容器和虽然看起来很像那么一回事儿，但是它的内核和网络等仍然是使用宿主机的。

如果我们想让这个容器拥有独立的网络呢？其实也很简单，只要给它一个桥接的接口就行了。首先我们需要准备一个桥接接口，此处以 netctl 为例，准备好配置文件：

Description="A bridge connection"
Interface=br0
Connection=bridge
BindsToInterfaces=(enp1s0 enp2s0 enp3s0 enp4s0)
IP=dhcp

这样就是将 enp1s0 ~ enp4s0 这四张网卡桥接，形成 br0 接口，并通过 DHCP 获得 IP 地址。

然后微调启动命令，增加一个命令 --network-bridge=br0 即可，例如：

`1`	`systemd-nspawn -b -D /var/lib/machines/ubuntu1604 --bind=/lib/firmware --network-bridge=br0`

启动完成后，通过 ip link 命令就能看到接口了，我这里看到的是 host0@if9。之后再按照对应的发行版修改一下网络配置，就完成了。

如果不想每次都通过这么长的命令来启动容器，我们可以使用 systemd 提供的 machinectl 命令来完成容器的开启与关闭，比如：

1
2
3

machinectl start ubuntu1604
machinectl login ubuntu1604
machinectl stop ubuntu1604

如果需要添加额外的参数，可以前往 /etc/systemd/nspawn 目录创建一个同名文件，例如 ubuntu1604.nspawn，在其中写上这个容器的配置，例如：

[Exec]
Boot=true

[Files]
Bind=/lib/firmware:/lib/firmware

[Network]
VirtualEthernet=yes
Private=yes
Bridge=br0

以上就是比较完整的容器创建和使用过程啦~

CentOS 稍微复杂一点，因为他没有直接提供最小的 rootfs，我们要自己从 ISO 中解压安装，整个过程如下：

# 切换到超级用户
sudo -s

# 建立一些文件夹
mkdir -p /var/lib/machines/centos7
mkdir -p /tmp/iso
mkdir -p /tmp/squashfs
mkdir -p /tmp/rootfs

# 挂载 CentOS 7 的系统盘
mount /mnt/Disk2/OS/Linux/CentOS-7-x86_64-Minimal-1708.iso /tmp/iso
mount /tmp/iso/LiveOS/squashfs.img /tmp/squashfs
mount /tmp/squashfs/LiveOS/rootfs.img /tmp/rootfs

# 复制系统文件，速度看你的硬盘，可能会比较慢_(:з」∠)_
cp -pr /tmp/rootfs/* /var/lib/machines/centos7

# 卸载一些不会再用到的镜像
umount /tmp/{rootfs,squashfs}

# 安装一下 yum
mkdir -p /var/lib/machines/centos7/mnt/iso
mount --bind /tmp/iso /var/lib/machines/centos7/mnt/iso

chroot /var/lib/machines/centos7 /usr/bin/rpm -ivh --nodeps /mnt/iso/Packages/rpm-4.11.3-25.el7.x86_64.rpm
chroot /var/lib/machines/centos7 /usr/bin/rpm -ivh --nodeps /mnt/iso/Packages/yum-3.4.3-154.el7.centos.noarch.rpm

# 配置一下基本系统，执行最小安装
echo "[cdrom]
name=Install CD-ROM
baseurl=file:///mnt/iso
enabled=0
gpgcheck=1
gpgkey=file:///mnt/iso/RPM-GPG-KEY-CentOS-7" > /var/lib/machines/centos7/etc/yum.repos.d/cdrom.repo

chroot /var/lib/machines/centos7 /usr/bin/yum --disablerepo=\* --enablerepo=cdrom -y reinstall yum
chroot /var/lib/machines/centos7 /usr/bin/yum --disablerepo=\* --enablerepo=cdrom -y groupinstall "Minimal Install"

# 删掉 ISO 源
rm /var/lib/machines/centos7/etc/yum.repos.d/cdrom.repo

# 卸载 ISO
umount /var/lib/machines/centos7/mnt/iso /tmp/iso

# 设置一下 root 密码之类的
chroot /var/lib/machines/centos7 /usr/bin/passwd root

# 进入虚拟环境，执行这段脚本
systemd-nspawn -D /var/lib/machines/centos7 --bind=/lib/firmware << _END_POSTINSTALL_
# 换源，先备份
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
curl https://lug.ustc.edu.cn/wiki/_export/code/mirrors/help/centos\?codeblock=3 > /etc/yum.repos.d/CentOS-Base.repo

# 更新
yum makecache
# 安装一个缺失的依赖，为什么会缺我也不知道…
yum install lvm2-libs -y

# 这些服务是作为一个容器不需要的，把他们关掉
systemctl disable auditd.service
systemctl disable kdump.service
systemctl disable multipathd.service
systemctl disable network.service
systemctl disable smartd.service
systemctl disable lvm2-monitor.service
systemctl disable sshd.service

# 设置 locale，如果需要中文就用 zh_CN.UTF-8
echo LANG=en_US.UTF-8 > /etc/locale.conf

# 设置主机名
echo CentOS-7 > /etc/hostname

_END_POSTINSTALL_

# 然后就可以开机了（（（
systemd-nspawn -b -D /var/lib/machines/centos7 --bind=/lib/firmware

参考资料：

Extend Storage with iSCSI and USB Gadget for Your PS4

Wed, 20 Sep 2017 00:00:00 +0000

After getting my hands on the PS4, I couldn’t help but buy a ton of games. I quickly ran out of space on the 500GB internal hard drive, so I decided to upgrade it to a 1TB drive. However, now even the 1TB drive is almost full, and I don’t want to go through the hassle of upgrading to a 2TB drive. Plus, I have no other use for the 1TB WD Black drive that I took out, so it’s quite frustrating.

After doing some research, I discovered that in a recent system update, the PS4 added a feature that allows you to format a USB storage as extended storage. This means you can use a USB 3.0 external hard drive to install games. I’ve always had some bias against using external hard drives, but this feature is actually a great way to expand the storage.

When it comes to expanding storage on a regular PC, there are many solutions available. For example, I could set up a shared folder using the SMB protocol, which is natively compatible with Windows and macOS. For Linux, all I need to do is load the cifs.ko module. Of course, using NFS is also a viable option. If encryption is required or if I need direct access to a raw disk, I could use iSCSI. Unfortunately, none of these methods are possible on a PS4, as it is currently not jailbroken.

So, I started thinking if it’s possible to expose some of these methods to the PS4 through USB. For example, imagine a device that acts as a USB device and has a Gigabit Ethernet interface, allowing it to access an iSCSI target on the local network. Then, through some magic, it would pass the iSCSI target’s disk device through to the USB, making the PS4 recognize it as a large storage.

To explore the feasibility of this idea, I bought an RK3399 board from Taobao. It comes with a USB Type-C interface and a Gigabit Ethernet interface. It also has decent CPU performance, although I haven’t been able to find a working Mali GPU driver yet. It supports both Android and Linux operating systems.

After I received the board, I found that it came pre-installed with both Android and Ubuntu dual systems. Without hesitation, I decided to reinstall the system. I had no need for Android, and I didn’t really want to use Ubuntu either, so I prepared an SD card and put the root file system of ARM64 Arch Linux ARM into it. Using Rockchip’s tool, I modified the content of the parameter partition, specifying the SD card as the root partition, and then restarted the device. After the boot process finished, I was able to install arch-install-scripts and completely remove Ubuntu from the eMMC. However, remember to backup the /system directory of Ubuntu, as I would still need the firmware in it later (I have no idea why it’s located at /system/etc/firmware instead of /etc/firmware; the modified Rockchip kernel is quite strange). If there is no need to manipulate U-boot, this should be sufficient. After restarting, I entered Arch Linux and performed basic configurations (network, pacman source, etc.). If you want to use the full-featured U-boot, you can refer to the information in this link to compile the u-boot provided by Rockchip yourself. The one provided by Firefly is heavily trimmed and basically unusable.

Then, I tried to figure out how to transform the USB-C interface into a fake USB drive. Initially, I thought I would need to write a kernel module, but after reading the documentation, I realized it was much simpler than that: the kernel exposes the configuration interface of the USB Gadget through a file system for users to manipulate. I just needed to create some files, make some links, and voila, I can accomplish it! I will put the code and comments together:

# First, we need to confirm if the configfs is mounted and the mount location.
mount | grep configfs
# Usually, it is mounted at /sys/kernel/config, so let's go there.
pushd /sys/kernel/config/usb_gadget
# Create a USB Gadget and add a function for mass storage.
mkdir g.1/functions/mass_storage.0 -p
# Wait for the kernel to complete the necessary work (essential in scripts).
sleep 1
# Write the device file that needs to be forwarded, or it can be a raw disk file in local filesystem. But be careful not to wear out the MMC (lol).
# Let's try using an SD card here.
echo "/dev/mmcblk0" > g.1/functions/mass_storage.0/lun.0/file
# Mark it as non-removable, i.e., not a USB flash drive.
echo 0 > g.1/functions/mass_storage.0/lun.0/removable

# Create some strings and ID information.
mkdir g.1/strings/0x409
mkdir -p g.1/configs/c.1/strings/0x409
echo 0xa4a2 > g.1/idProduct
echo 0x0525 > g.1/idVendor
echo 1234567890 > g.1/strings/0x409/serialnumber
echo ntzyz > g.1/strings/0x409/manufacturer
echo "iSCSI over USB" > g.1/strings/0x409/product

# Create a configuration and link the previously set functions.
echo "cnf1" > g.1/configs/c.1/strings/0x409/configuration
ln -s g.1/functions/mass_storage.0 g.1/configs/c.1

# Specify the controller used by this Gadget, you can confirm it by using the command "find /sys -name gadget".
echo fe800000.dwc3 > g.1/UDC

popd

Take out your USB 3.0 Type-C cable, connect RK3399 to your computer, and then run this script. If everything goes well, you should be able to see the new hardware, and the kernel will probably have these outputs:

[   52.614900] fusb302 4-0022: PD disabled
[   52.617919] cdn-dp fec00000.dp: [drm:cdn_dp_pd_event_work] Not connected. Disabling cdn
[   52.628791] rockchip-dwc3 usb@fe800000: USB peripheral connected
[   52.636063] android_work: did not send uevent (0 0           (null))
[   52.649973] android_work: sent uevent USB_STATE=CONNECTED
[   52.679112] configfs-gadget gadget: super-speed config #1: c
[   52.679840] android_work: sent uevent USB_STATE=CONFIGURED

It’s done! Isn’t it simple?

Next, let RK3399 access a remote disk. Of course, you can also add a SATA hard drive to RK3399, but it seems more convenient to use a portable hard drive directly (lol). In principle, you can use NFS/CIFS or similar network file systems to access a partition on NAS, then DD out a large sparse disk image, format it, and use this file directly as a disk. However, when I tried this, writing a 1GB animation file initially surged to 300MB/s but dropped to 0 within two seconds, and I saw a kernel panic error from RK3399’s serial output.

Indeed, in this case, we should use iSCSI! To enable iSCSI target service on Windows, you can choose to switch to a Server version of Windows, which comes with iSCSI target server functionality. If you can’t afford it or don’t want to switch, you can use some user-space software to implement it, such as the free StarWind for personal users. However, it seems that StarWind does not support creating sparse img files, which is a bit annoying. On Linux, you can refer to the Open iSCSI section on the Arch Wiki for information. I won’t go into details about configuring the iSCSI target service here; it’s relatively simple, just use CHAP for user authentication, no need for complexity.

Regarding RK3399, first send a sendtargets request to the iSCSI server:

`1`	`iscsiadm -m discovery -t sendtargets -p 10.10.28.75`

Once you get the target list, you can log in:

`1`	`iscsiadm -m node --targetname=iqn.2008-08.com.starwindsoftware:ntzyz-gen8-ps4volume --login`

As expected, there will be a lot of kernel logs output at this time, such as finding sda. At this point, you have successfully created a connection with the server, and all changes on sda will be written to the remote disk. As long as you use this device file as the file corresponding to lun.0 of Mass Storage in Gadget, our ultimate goal of a USB interface remote disk is achieved!

Unplug the USB from the computer, plug it into the PS4 console, go to Settings -> Peripheral Devices -> USB Devices, and you will see a mobile hard drive named iSCSI over USB with the manufacturer ntzyz. Format it as extended storage, and then you can use it. After testing, moving NieR Automata to this extended storage and playing it did not show any noticeable difference in user experience.

Fix HiDPI issue for Netease Cloud Music

Thu, 07 Sep 2017 00:00:00 +0000

网易云音乐 Linux 版推出很久了，肯定已经有很多人反馈过 HiDPI 下不能正常工作的问题了，然而似乎网易生娃不养娃，版本号 1.0.0 从一开始就没有变过（

在网上搜索 HiDPI 下字太小难以阅读就能找到最简单的解决方案：添加参数 –force-device-scale-factor=2 就能让他强制两倍渲染。不过这个方法在我这儿（ArchLinux，KDE）上并不是什么好的解决方案：启动程序时，程序按照环境 DPI 创建了窗体，但是内容部分则依然使用原分辨率渲染，效果就是只有坐上 1/4 能正常显示内容，剩下的都是白色的空白。只要我的全局 DPI 不是 100%，这些空白就一直存在。

既然网易支持强制缩放倍数，只要我让他认为自己运行的环境是 100% DPI 即可，解决方案如下：

1
2
3

#!/bin/bash

QT_SCREEN_SCALE_FACTORS=1 /usr/bin/netease-cloud-music --force-device-scale-factor=2

为了方便使用，我们可以在 /usr/local/bin/ 下创建同名脚本，执行后就可以直接 pin 到任务栏了。

9 月 29 日更新

由于某种神秘力量我滚完 Arch Linux 后这个网易云音乐又特么出问题了：运行后什么输出都没有，然后就没有然后了。看了一下 AUR 的评论得知需要添加参数 --no-sandbox，所以脚本修改如下：

1
2
3

#!/bin/bash

QT_SCREEN_SCALE_FACTORS=1 /usr/bin/netease-cloud-music --force-device-scale-factor=2 --no-sandbox

Implementing a Web Terminal

Sat, 26 Aug 2017 00:00:00 +0000

说起来也是有趣，本来是研究一下 WebSocket 准备给论坛/博客增加实时更新之类的特性，结果看着看着就脑洞大开搞了这么个玩意儿（（

首先明确一下，这里说的 Web Terminal 是指再网页中实现的，类似于终端模拟器的玩意儿。举例的话应该是类似于 Linode 的 LiSH 和 Visual Studio Code 中内置的那个终端，而不是 ConoHa 提供的 VNC 式的终端（其实那玩意儿是个远程桌面了）。最终目标的效果就是和 Secure Shell 类似：打开一个网页，就能启动一个网页所在服务器的 shell，比如到处都有的 bash 或者非常强大的 zsh，然后就可以与这个终端进行交互式的操作，比如使用 vim 编辑文件，或者查阅 man 中的手册。

让我们从最简单的一些需求开始，如果只是需要远程执行一些命令或者脚本，那么我们只需要任何一个能调用系统 shell 的编程语言就行了。这里以 node 为例，代码很简单：

'use strict';
const express = require('express');
const child_process = require('child_process');
let server = express();

server.get('/eval', (req, res) => {
  if (req.query.cmd) {
    child_process.exec(req.query.cmd, (err, stdout, stderr) => {
      res.send({
        status: 'ok',
        stdout,
        stderr,
      });
    })
  } else {
    return res.send({
      status: 'ok';
    })
  }
});

server.listen(8123, 'localhost', () => {
 console.log(`Server running at http://localhost:8123`);
});

安装好 express 后，执行这段代码，打开另一个终端，执行代码：

`1`	`curl 'http://localhost:8123/eval?command=ls'`

我们就能看到 node 所在目录的文件列表了。看起来不错，但是如果需要执行面向终端的程序呢？

面向终端的程序，顾名思义，这种程序需要控制一个终端，同时有能力进行 job-control（fg 等）和终端相关的信号（SIGINT 等）。典型的面向终端的程序就有：nano、vim、htop、less，等等。要让这些程序执行，我们需要通过 POSIX 的接口来创建一个伪终端（pseudoterminal，简称 pty）。

伪终端由两个虚拟的设备组成：一个 pseudoterminal master 和一个 pseudoterminal slave（pts）。这两个虚拟设备之间可以互相通讯，类似一个串口设备。两个进程可以打开这两个虚拟设备进行通讯，类似于一个管道。伪终端的关键就是 pts，这个设备在操作上和真实的终端设备（tty1, ttyS1, …）基本一致，不同之处在于 pts 没有速率之类的属性。所有能在 tty 上使用的操作都能在 pts 上使用，不支持的部分属性会被自动忽略，反正没什么卵用（（

知道这些东西之后，终端模拟器的工作原理就很简单了：终端模拟器创建了一对 pty 设备，同时在 pty slave 上启动当前用户的默认 shell，比如 execlp("/usr/bin/zsh", [ “–login” ])。pts 会将程序所有的输出发送给 pty master，终端模拟器在拿到这些数据后，再按照指定终端的标准将其输出。同时，所有的键盘输入也会发送给 pty slave。大致就是如下：

+----------+
| X Server |
+----+-----+
     |
+----+--------------+  +------------+
| Terminal Emulator +--+ pty master +
+-------------------+  +--+-----+---+
                          |     |
                       +--+-----+--+
                       + pty slave +
                       +--+-----+--+
                          |     |
        +-----------------+-----+---+
        + Terminal-oriented program |
        +---------------------------+

Secure Shell 的远程登录的原理同样类似：ssh 客户端首先和 sshd 协商加密，互相认证，然后建立一个 SSH channel，由服务端创建一对 pty，然后将 pty master 的输出放到 SSH channel 中。ssh 客户端与服务端之间通过 SSH channel 通讯，便实现了远程登陆。

那么，Web Terminal 的实现思路就很明确了：在浏览器上，我们需要找到一个比较好用的终端框架（或者自己撸一个），在服务器上，我们需要一个当前程序语言与 ptmx 的接口（或者自己撸一个）。而通讯方面，SSH 用的是 TCP，Web 上能用的也就是 WebSocket 了（除非你想 XMLHttpRequest 然后疯狂刷新），这里能找到框架最好，全都自己撸就太累了（

嘛。虽然 npm 上坑爹的包非常多，但是在这种时候基本上还是能做到想要啥就有啥的。这里我选择了 xterm.js 作 HTML5 中的终端组件，node-pty 做服务端的 pty 操作工具。这两个也正是 Visual Studio Code 中内置的终端所采用的依赖。WebSocket 方面，我选择了 Socket.IO 这个框架。当然，为了让 ES6 Module 正常工作，我们还需要用 webpack 来处理。依靠着强大的 xterm.js 和 node-pty，需要我们来完成的工作非常少。以下晒代码：

服务端 JavaScript ：

const express = require("express");
const site = express();
const http = require("http").Server(site);
const io = require("socket.io")(http);
const net = require("net");
const pty = require("node-pty");

site.use("/", express.static("."));

io.on("connection", function (socket) {
  let ptyProcess = pty.spawn("bash", ["--login"], {
    name: "xterm-color",
    cols: 80,
    rows: 24,
    cwd: process.env.HOME,
    env: process.env,
  });
  ptyProcess.on("data", (data) => socket.emit("output", data));
  socket.on("input", (data) => ptyProcess.write(data));
  socket.on("resize", (size) => {
    console.log(size);
    ptyProcess.resize(size[0], size[1]);
  });
});

http.listen(8123);

浏览器端 JavaScript：

import Terminal from "xterm";
import "xterm/src/xterm.css";
import io from "socket.io-client";

Terminal.loadAddon("fit");

const socket = io(window.location.href);

const term = new Terminal({
  cols: 80,
  rows: 24,
});
term.open(document.getElementById("#terminal"));
term.on("resize", (size) => {
  socket.emit("resize", [size.cols, size.rows]);
});

term.on("data", (data) => socket.emit("input", data));

socket.on("output", (arrayBuffer) => {
  term.write(arrayBuffer);
});

window.addEventListener("resize", () => {
  term.fit();
});
term.fit();

Webpack 配置：

const path = require("path");

module.exports = {
  entry: "./src/entry.js",
  output: {
    path: path.join(__dirname, "dist"),
    filename: "bundle.js",
  },
  module: {
    loaders: [{ test: /\.css$/, loader: "style-loader!css-loader" }],
  },
};

运行效果：

完整的代码可以参考 GitHub 上的 playground 仓库。

Dynamically Evaluate JavaScript on Single-Page-Application

Sun, 13 Aug 2017 00:00:00 +0000

之前在写自己的博客框架的时候遇到了一个问题：文章中的 script 标签没有任何作用，以 Vue 为 MVVM 框架举个例子：

<body>
  <div id="app">
    <p v-html="html"></p>
  </div>
  <script>
    var vm = new Vue({
      el: "#app",
      data: {
        html: '<script> console.log("JavaScript is awesome"); <\/script>',
      },
    });
  </script>
</body>

这段代码的意思很简单，创建一个 ID 为 app 的 DIV，其中包含一个 p 标签，vm 实例化之后，p 的内容就是 data 字段的 html，渲染结束后效果应该是这样：

<body>
  <div id="app">
    <p>
      <script>
        console.log("JavaScript is awesome");
      </script>
    </p>
  </div>
  <script>
    // emmmm
  </script>
</body>

好吧……看起来这代码很不规矩，但是浏览器应该是接受在任意位置出现的 script 标签的，也就是按照预期，我们会在控制台里看到 JavaScript is awesome 这句话，没毛病。

然而实际结果是……并没有。原因很简单，v-html 是通过 innerHTML 来实现的，而 HTML5 的标准规定了通过 innerHTML 得到的 script 标签不会被执行。此举的目的是为了防止利用这个特性来进行 XSS 攻击，但是依然有不少场景需要使用这些脚本，所以我们需要一些 trick 来让浏览器加载这些脚本。与此同时，如果 script 是通过 appendChild 添加到 DOM，这段脚本是能被正常执行的。

所以解决方案也就很简单了：找到需要执行的 script 标签，然后通过 createElement 创建空 script，并将待执行标签的数据复制到空 script 中（比如 src，或者 innerHTML），最后通过 appendChild 将它添加到 DOM 中去。

其中，找到 script 标签的方法比较多，比如可以通过正则对字符串进行匹配。不过考虑到通过 innerHTML 设置的 script 标签仅仅只是没有执行，他们仍然在 DOM 树中，所以 querySelector 在鲁棒性（划掉）上就比正则高出一截，代码如下：

<body>
  <div id="app">
    <p v-html="html"></p>
  </div>
  <script>
    var vm = new Vue({
      el: "#app",
      data: {
        html: '<script> console.log("JavaScript is awesome"); <\/script>',
      },
      created: function () {
        // 等待 DOM 刷新
        this.$nextTick(function () {
          let app = document.querySelector("#app");
          let scripts = Array.from(app.querySelectorAll("script"));
          scripts.forEach(function (script) {
            let node = document.createElement("SCRIPT");
            let src = script.getAttribute("src");
            if (src) {
              node.setAttribute("src", src);
            } else {
              node.innerHTML = script.innerHTML;
            }
            document.querySelector("body").appendChild(node);
          });
        });
      },
    });
  </script>
</body>

这样，我们就能顺利地在控制台里看到输出的字符串啦（

当然这样做也有他的缺点：

使用了全局的 JavaScript 环境。如果代码中出现了一些声明（比如用 let 定义了变量），或者是通过 new Audio() 之类的方式播放了音频，即使你在适当的时候将他们从 DOM 树中移除，他们产生的影响仍然将保留在当前的上下文中。我个人的解决方案是，使用一个 flag 来记录是否添加过 script，若添加过，则后续的路由操作将在 beforeEach 的钩子处被劫持，并将 fullPath 直接应用到 window.location.href 上，强制刷新当前页面，来获得一个全新的 JavaScript 上下文。
没有充分模拟 script 标签的特性。按照正常的网页加载模式，浏览器在加载和执行 script 标签时是阻塞的，顺序执行的。通过这种方式添加的 script 执行顺序并没有任何明确的规定，可能导致出现依赖上的问题。

Implementing AES-128-ECB (School Homework)

Wed, 17 May 2017 00:00:00 +0000

首先呢这份代码是这学期选修课的大作业，前后也花了点心思，于是还是贴到这儿，顺便给博客除除草 ww

作业的题目有两大类，一类是使用常见的对称/非对称加密算法（DES/AES/RC5/RSA/Blowfish/blabla）实现一个可用的加解密工具，另一个则是实现一个 HASH 程序，类似于 md5sum 或者是 sha256sum 这种。我选择了做 AES 主要是有几个原因：

老师说了，杂凑算法于加解密算法比较起来，实现难度低，因此基础成绩就会相对低一点（然而我觉得两个都蛮简单的啊）。
在上课之前就已经接触过 AES 这个算法了（归功于 Shadowsocks 啦），用了这么久的东西，也算是对它的实现比较感兴趣的（然而怎么想都知道自己写的破代码性能会被 OpenSSL 吊着打）。

当然按照老师的说法，简单的实现只是最基本的要求，所以我这里打算做三分实现，分别是 CPU 单线程，CPU 多线程和 GPGPU。其实这里面核心代码都是完全一样的，无非就是分组加密和密钥更新上有点区别，但是听起来就高大上了很多（没毛病）。

AES 本身一次只能使用 128bit 的密钥对 128bit 的数据进行加密，所以在实际中必须使用分组加密算法来完成大文件的处理。往简单里讲，分组加密不过是把数据分成等长的分片，然后每次对一个分片进行加密，然后在将加密后的分片组合成文件。解密过程与之类似。这种最简单粗暴地分组加密被称为电子密码本（ECB）模式。然而实际上，由于相同分片数据使用相同的密钥进行加密后，输出必然也是相同的，这样也就会使加密的内容带有特征，存在潜在的隐患，于是人们提出一些会修改后续分组的密钥的分组加密算法，比如密码分组链接模式（CBC）、密文反馈模式（CFB）和计数器模式（CTR）。CBC 和 CFB 在设计上，下一分片的密钥受到上一分片的影响，也就意味着这个算法必须线性的对密码进行加密，不利于多线程的实现。那么可选的就是 CTR 和 ECB 模式了。二选一的话，选择 ECB 而不是 CTR 主要是因为我懒，ECB 只要一个函数改变传入指针就能跑了（

分区模式敲定之后，就轮到 AES 算法的了解和实现了。整个算法主要由两部分组成：数据变换和密钥拓展。

首先实现一份密钥拓展函数，其算法参考：Rijndael key schedule – Wikipedia，使用上很简单，直接将密钥指针传入即可，返回的是堆上分配的结果，用完记得要 free（（

extern uint8_t round_const[];
uint8_t *key_schedule(const uint8_t *input_key) {
    uint8_t *output = (uint8_t *)malloc((10 + 1) * 16 * sizeof(uint8_t));
    uint8_t temp[4], *cursor_4_before, *cursor_16_before, *cursor_current;
    size_t counter, i, j;

    // Copy the first 16 bytes
    memcpy(output, input_key, 16 * sizeof(uint8_t));

    // Initialize cursors.
    cursor_current = output + 16;
    cursor_4_before = cursor_current - 4;
    cursor_16_before = cursor_current - 16;

    // Rock and roll.

    for (counter = 0; counter != 10; counter++) {
        // RotWord
        temp[3] = cursor_4_before[0];
        temp[0] = cursor_4_before[1];
        temp[1] = cursor_4_before[2];
        temp[2] = cursor_4_before[3];

        // SubBytes and add
        for (i = 0; i != 4; ++i) {
            cursor_current[i] = sbox[temp[i]] ^ cursor_16_before[i];
            if (i == 0) {
                cursor_current[i] ^= round_const[counter];
            }
        }

        // Forward all cursors
        cursor_current += 4, cursor_4_before += 4, cursor_16_before += 4;

        // Three step remaining is simple.
        for (i = 0; i != 3; ++i) {
            for (j = 0; j != 4; ++j) {
                cursor_current[j] = cursor_4_before[j] ^ cursor_16_before[j];
            }
            cursor_current += 4, cursor_4_before += 4, cursor_16_before += 4;
        }
    }

    return output;
}

AES 128 会对输入数据进行十轮处理，其中 AddRoundKey 步骤使用的密钥就是由上面函数生成的。

AES 128 的十轮处理中，必然会有的四个步骤：字节替换、行移位、列混淆和密钥相加（XOR）。

字节替换提供了加密法非线性的变换能力，实现起来相当容易：将每一个元素代入 S 盒中即可：

extern uint8_t sbox[];
void sub_bytes(uint8_t *input) {
    int i = 15;
    do {
        input[i] = sbox[input[i]];
    } while (--i >= 0);
}

这里补充一下，AES 对输入的明文和密钥的处理有一点反人类，比如明文 {a0, a1, a2, a3, …, a15}，我们要把它看成这样的一个矩阵：

a0  a4  a8  a12
a1  a5  a9  a13
a2  a6  a10 a14
a3  a7  a11 a15

所以上面给出的 OFFSET 宏乍一看比较反人类。。对应的，行移位后的矩阵应该是这样的：

a0  a4  a8  a12
a5  a9  a13 a1
a10 a14 a2  a6
a15 a3  a7  a11

在内存里也就是这样的数组： {a0, a5, a10, a15, a4, a9, a14, a3, …}

剩下两个函数就是列混淆和密钥相加。其中列混淆可以视为 Rijndael 有限域之下的矩阵乘法，详情可以看 Wikipedia，或者是翻翻书。这里我提前打了个表，来加速完成。变量名叫 cache 是本来想实时计算缓存结果的，最后还是干脆直接打全了。。

static inline uint8_t mul2(uint8_t a) {
    static uint8_t cache[256] = {
        0x00, 0x02, 0x04, 0x06, 0x08, 0x0A, 0x0C, 0x0E, 0x10, 0x12, 0x14, 0x16, 0x18, 0x1A, 0x1C, 0x1E,
        0x20, 0x22, 0x24, 0x26, 0x28, 0x2A, 0x2C, 0x2E, 0x30, 0x32, 0x34, 0x36, 0x38, 0x3A, 0x3C, 0x3E,
        0x40, 0x42, 0x44, 0x46, 0x48, 0x4A, 0x4C, 0x4E, 0x50, 0x52, 0x54, 0x56, 0x58, 0x5A, 0x5C, 0x5E,
        0x60, 0x62, 0x64, 0x66, 0x68, 0x6A, 0x6C, 0x6E, 0x70, 0x72, 0x74, 0x76, 0x78, 0x7A, 0x7C, 0x7E,
        0x80, 0x82, 0x84, 0x86, 0x88, 0x8A, 0x8C, 0x8E, 0x90, 0x92, 0x94, 0x96, 0x98, 0x9A, 0x9C, 0x9E,
        0xA0, 0xA2, 0xA4, 0xA6, 0xA8, 0xAA, 0xAC, 0xAE, 0xB0, 0xB2, 0xB4, 0xB6, 0xB8, 0xBA, 0xBC, 0xBE,
        0xC0, 0xC2, 0xC4, 0xC6, 0xC8, 0xCA, 0xCC, 0xCE, 0xD0, 0xD2, 0xD4, 0xD6, 0xD8, 0xDA, 0xDC, 0xDE,
        0xE0, 0xE2, 0xE4, 0xE6, 0xE8, 0xEA, 0xEC, 0xEE, 0xF0, 0xF2, 0xF4, 0xF6, 0xF8, 0xFA, 0xFC, 0xFE,
        0x1B, 0x19, 0x1F, 0x1D, 0x13, 0x11, 0x17, 0x15, 0x0B, 0x09, 0x0F, 0x0D, 0x03, 0x01, 0x07, 0x05,
        0x3B, 0x39, 0x3F, 0x3D, 0x33, 0x31, 0x37, 0x35, 0x2B, 0x29, 0x2F, 0x2D, 0x23, 0x21, 0x27, 0x25,
        0x5B, 0x59, 0x5F, 0x5D, 0x53, 0x51, 0x57, 0x55, 0x4B, 0x49, 0x4F, 0x4D, 0x43, 0x41, 0x47, 0x45,
        0x7B, 0x79, 0x7F, 0x7D, 0x73, 0x71, 0x77, 0x75, 0x6B, 0x69, 0x6F, 0x6D, 0x63, 0x61, 0x67, 0x65,
        0x9B, 0x99, 0x9F, 0x9D, 0x93, 0x91, 0x97, 0x95, 0x8B, 0x89, 0x8F, 0x8D, 0x83, 0x81, 0x87, 0x85,
        0xBB, 0xB9, 0xBF, 0xBD, 0xB3, 0xB1, 0xB7, 0xB5, 0xAB, 0xA9, 0xAF, 0xAD, 0xA3, 0xA1, 0xA7, 0xA5,
        0xDB, 0xD9, 0xDF, 0xDD, 0xD3, 0xD1, 0xD7, 0xD5, 0xCB, 0xC9, 0xCF, 0xCD, 0xC3, 0xC1, 0xC7, 0xC5,
        0xFB, 0xF9, 0xFF, 0xFD, 0xF3, 0xF1, 0xF7, 0xF5, 0xEB, 0xE9, 0xEF, 0xED, 0xE3, 0xE1, 0xE7, 0xE5,
    };
    return cache[a];
}
void mix_columns(uint8_t *input) {
    size_t i;
    uint8_t temp[16], tmp;

    memcpy(temp, input, 16);

    for (i = 0; i != 16; i += 4) {
        tmp = temp[i] ^ temp[i + 1] ^ temp[i + 2] ^ temp[i + 3];
        input[i + 0] = mul2(temp[i + 0] ^ temp[i + 1]) ^ temp[i + 0] ^ tmp;
        input[i + 1] = mul2(temp[i + 1] ^ temp[i + 2]) ^ temp[i + 1] ^ tmp;
        input[i + 2] = mul2(temp[i + 2] ^ temp[i + 3]) ^ temp[i + 2] ^ tmp;
        input[i + 3] = mul2(temp[i + 3] ^ temp[i + 0]) ^ temp[i + 3] ^ tmp;
    }
}

最后是密钥相加。太简单了对应位置 Xor 就行了，甚至不用单独写个函数。将这以上四部分组合起来之后，只要写一个函数对明文进行十轮加密，函数大概这样：

void aes_128_single_block(const uint8_t *input, uint8_t *round_keys, uint8_t *output) {
    size_t i, j;
    uint8_t *result, *cursor_round_key;

    // initialize cursors.
    result = output;
    cursor_round_key = round_keys;

    // Copy plain text.
    for (i = 0; i != 16; ++i) {
        result[i] = input[i] ^ cursor_round_key[i];
    }
    cursor_round_key += 16;

    // Run rounds excluding the last round.
    for (i = 0; i != 9; ++i) {
        sub_bytes(result);
        shift_rows(result);
        mix_columns(result);
        // Add round
        for (j = 0; j != 16; ++j) {
            result[j] = result[j] ^ cursor_round_key[j];
        }
        cursor_round_key += 16;
    }

    // Now the last round.
    sub_bytes(result);
    shift_rows(result);
    for (i = 0; i != 16; ++i) {
        result[i] = result[i] ^ cursor_round_key[i];
    }
}

其中 round_keys 就是最开头给的函数生成的带拓展的密钥。

完成以上的加密相关代码后，只要小幅改动部分代码就能实现密文的解密了。比如 AES 里四个步骤的对应函数，除了轮密钥相加以外，其他三个函数都需要编写对应的反函数（姑且这么叫吧）。至于轮密钥相加以外，本质工作就是将轮密钥和体（state）进行异或，根据异或的性质，两次异或直接能就是本身，所以没有再去折腾了。

void aes::inverse_sub_bytes(byte *input) {
    int i = 15;
    do {
        input[i] = inverse_sbox[input[i]];
    } while (--i >= 0);
}

void inverse_shift_rows(byte *input) {
    int index, inner_index;
#define OFFSET(i, j) ((j) * 4 + (i))
    for (index = 0; index != 4; ++index) {
        byte temp;
        inner_index = index;
        while (inner_index-- > 0) {
            temp = input[OFFSET(index, 3)];
            input[OFFSET(index, 3)] = input[OFFSET(index, 2)];
            input[OFFSET(index, 2)] = input[OFFSET(index, 1)];
            input[OFFSET(index, 1)] = input[OFFSET(index, 0)];
            input[OFFSET(index, 0)] = temp;
        }
    }
#undef OFFSET
}

void aes::inverse_mix_columns(byte *input) {
    size_t i;
    byte temp[16], tmp;

    memcpy(temp, input, 16);

    for (i = 0; i != 16; i += 4) {
        tmp = temp[i] ^ temp[i + 1] ^ temp[i + 2] ^ temp[i + 3];
        input[i + 0] = mul2(temp[i + 0] ^ temp[i + 1]) ^ temp[i + 0] ^ tmp;
        input[i + 1] = mul2(temp[i + 1] ^ temp[i + 2]) ^ temp[i + 1] ^ tmp;
        input[i + 2] = mul2(temp[i + 2] ^ temp[i + 3]) ^ temp[i + 2] ^ tmp;
        input[i + 3] = mul2(temp[i + 3] ^ temp[i + 0]) ^ temp[i + 3] ^ tmp;

        byte u, v;
        u = mul2(mul2(temp[i + 0] ^ temp[i + 2]));
        v = mul2(mul2(temp[i + 1] ^ temp[i + 3]));
        tmp = mul2(v ^ u);

        input[i + 0] ^= tmp ^ u;
        input[i + 2] ^= tmp ^ u;
        input[i + 1] ^= tmp ^ v;
        input[i + 3] ^= tmp ^ v;
    }
}

最后的解密过程就是加密过程反着来，写起来也没什么困难：

void aes::aes_decrypt_single_block(const byte *input, byte * const round_keys, byte *output) {
    size_t i, j;
    byte *result, *cursor_round_key;

    // initialize cursors.
    result = output;
    cursor_round_key = round_keys + 160;

    for (i = 0; i != 16; ++i) {
        result[i] = input[i];
    }

    // The last round
    for (i = 0; i != 16; ++i) {
        result[i] = input[i] ^ cursor_round_key[i];
    }

    inverse_shift_rows(result);

    inverse_sub_bytes(result);

    cursor_round_key -= 16;

#ifdef DEBUG
    printf("# InvRound %d\n", 10);
    print_key(result);
#endif

    // Run rounds excluding the last round.
    for (i = 0; i != 9; ++i) {
        for (j = 0; j != 16; ++j) {
            result[j] = result[j] ^ cursor_round_key[j];
        }
        inverse_mix_columns(result);
        inverse_shift_rows(result);
        inverse_sub_bytes(result);
        cursor_round_key -= 16;
#ifdef DEBUG
        printf("# InvRound %d\n", 10 - i);
        print_key(result);
#endif
    }

    // Now the last addRoundKey.
    for (i = 0; i != 16; ++i) {
        result[i] = result[i] ^ cursor_round_key[i];
    }
}

以上就是 AES 的所有核心代码了，剩下的就是如何去调用这些函数。多线程方面，依靠 C++ 的 STL 库能很容易地实现线程的创建和锁控制。首先准备一个函数，将 key_schedule 和 aes 加密/解密函数包裹在一起。再准备一个函数按照指定步长去对输入指针内容进行加密/解密操作，代码很简单：

void process_unit(uint8_t *data_in, uint8_t *data_out, uint8_t *key, bool isEncrypt) {
    aes_cpu::byte *rkey = aes_cpu::aes::key_schedule(key);
    if (isEncrypt) {
        aes_cpu::aes::aes_128_single_block(data_in, rkey, data_out);
    }
    else {
        aes_cpu::aes::aes_decrypt_single_block(data_in, rkey, data_out);
    }
    delete[] rkey;
}

void thread_entry(uint8_t *data_in, uint8_t *data_out, uint8_t *key, size_t file_size, size_t current, size_t total, bool isEncrypt) {
    for (uint32_t i = current; i < file_size / 16; i += total) {
        if (i % 10000 == 0 && current == 0) {
            printf("process: %.2lf\r", (double)i * 100 / (file_size / 16));
        }
        process_unit(data_in + 16 * i, data_out + 16 * i, key, isEncrypt);
    }
}

现在只需要根据用户传入的线程数，创建若干线程即可完成工作：

bool isEncrypt = true;
std::vector<std::thread> threads(thread_count);

// Construct all threads, and let them go.
for (int i = 0; i != threads.size(); ++i) {
    threads[i] = std::thread(thread_entry, data_in, data_out, key, full_block_size, i, threads.size(), isEncrypt);
}

// Sync for all threads.
for (int i = 0; i != threads.size(); ++i) {
    threads[i].join();
}

涉及到任意文件加的时候就会遇到一个问题：文件大小不一定是 128 比特的整数倍。这时候就需要使用一些密码学上的填充算法。我简单的看了一下 PKCS#7 下的填充方法，简单说来就是：

若文件恰好为 16 字节的整数倍，后面强行补十六位；
填充内容为当前大小与填充后大小的差，比如当前是 126 字节，就补充两个 0x02。对于上方的特例，就补充 16 个 0x16；

这里就不把函数列出来了，一个 realloc 一个 for 就能解决。这样我们的多线程 AES 加解密就已经全部完成了，接下来就是将以上代码迁移到 CUDA 平台。

从整体上看，CUDA 和 CPU 的代码基本一致（废话都是 AES 能不一样有鬼了），主要的区别就在于调用 GPU 是一个类似于 Master 和 Slave 通讯的过程。在完成 PKCS#7 的填充后，我们要把明文，密钥等复制到显存，同时在显存里分配一块缓冲区来保存密文，代码大致如下：

uint8_t *round_keys = key_schedule(keys);
uint8_t *gpu_input = nullptr;
uint8_t *gpu_round_keys = nullptr;
uint8_t *gpu_output = nullptr;
cudaError_t cudaStatus;

#define ASSERT(s) \
    if (cudaStatus != cudaSuccess) { \
        fprintf(stderr, s"\n"); \
        goto Error; \
    }

// Select which GPU to work with.
cudaStatus = cudaSetDevice(0);
ASSERT("cudaSetDevice");

// Prepare GPU memory block
cudaStatus = cudaMalloc((void**)&gpu_input, size * sizeof(uint8_t));
ASSERT("cudaMalloc");
cudaStatus = cudaMalloc((void**)&gpu_round_keys, (11 * 16) * sizeof(uint8_t));
ASSERT("cudaMalloc");
cudaStatus = cudaMalloc((void**)&gpu_output, size * sizeof(uint8_t));
ASSERT("cudaMalloc");

// Copy data from memory to GPU memory.
cudaStatus = cudaMemcpy(gpu_input, input, size * sizeof(uint8_t), cudaMemcpyHostToDevice);
ASSERT("cudaMemcpy: gpu_input");
cudaStatus = cudaMemcpy(gpu_round_keys, round_keys, (11 * 16) * sizeof(uint8_t), cudaMemcpyHostToDevice);
ASSERT("cudaMemcpy: gpu_round_keys");

然后就可以启动 GPU 上的函数了：

size_t offset = 0;
const size_t threadsPerBlock = 512;
const size_t blockSize = 16;
while (offset * threadsPerBlock * blockSize * 16 < size) {
    if (offset % 100 == 0) {
        printf("progress: %lf\r", offset * (double)threadsPerBlock * blockSize * 1600 / size);
    }
    if (isEncrypt) {
        aes_128_single_block <<<blockSize, threadsPerBlock >>>(gpu_input, gpu_round_keys, gpu_output, size / 16, offset, threadsPerBlock * blockSize);
    }
    else {
        aes_decrypt_single_block <<<blockSize, threadsPerBlock >>>(gpu_input, gpu_round_keys, gpu_output, size / 16, offset, threadsPerBlock * blockSize);
    }
    offset++;
}

最后，把显存上的密文复制回内存，再写入到文件，就完成了整个加密过程：

cudaStatus = cudaGetLastError();
ASSERT("cudaGetLastError");
cudaStatus = cudaDeviceSynchronize();
ASSERT("cudaDeviceSynchronize");
cudaStatus = cudaMemcpy(output, gpu_output, size * sizeof(uint8_t), cudaMemcpyDeviceToHost);
ASSERT("cudaMemcpy");

Compiling My Own OpenWrt (x86_64)

Tue, 28 Feb 2017 00:00:00 +0000

OpenWrt 是一个十分著名的嵌入式 Linux 发行版，被广泛使用在家庭路由器的魔改上。其前身是 Linksys 被迫开源的 WRT54G 固件。最近几天买了一个 Intel J1900 CPU 的板子，集成了四张千兆网卡，于是就打算拿来跑 OpenWrt x86。但是呢，OpenWrt 官方放出的 x86_64 固件中使用的标准库是 uClibc，一个面向嵌入式 Linux 系统的小型的 C 标准库。可是手里的硬件确实正儿八经的 BayTrail CPU，使用 uClibc 未免感觉太憋屈了。不仅如此，绝大多数编译好的二进制软件都是使用 GNU C Library，同时 uClibc 没有提供对 glibc 的兼容，所以即便我是 x86 的 CPU，所有软件也都需要使用 OpenWrt 重新编译，很繁琐。于是，打算重新编译自己的 OpenWrt。

首先是获得源码，需要注意的是 git.openwrt.com 上的代码已经很久没有更新了，所有代码都应该从 GitHub 上取得。

1
2

git clone -b v15.05.1 https://github.com/openwrt/openwrt.git
cd openwrt

当然还要准备一下依赖：

`1`	`sudo apt-get install build-essential subversion libncurses5-dev zlib1g-dev gawk gcc-multilib flex git-core gettext libssl-dev`

然后更新 OpenWrt 的软件列表：

1
2

./scripts/feeds update -a
./scripts/feeds install -a

并生成默认配置：

`1`	`make defconfig`

然后就可以开始配置了：

`1`	`make menuconfig`

这里面大多数内容都没什么好说的，如果不明确就去搜索。不过强烈推荐激活 Global build settings 中的 Select all kernel module packages by default，软件没编译可以随时 make menuconfig && make，但是内核模块如果重新编译就不得不将板子上已经部署好的 OpenWrt 完全替换，因为每一次编译内核和内核模块都是与之前不兼容的。当然 CPU 和 SSD 足够大的话你顺手选上 Select all userspace packages by default 是最好的。至于 C 标准库的选择，可以在 Advanced configuration options > Toolchain Options > C Library implementation 中修改。如果你选择了很多包内置到 rootfs 时，默认的 IMG 文件可能不够用，此时你就需要修改 Target Images 中的 Root filesystem partition size，和 ext4 中的 Maximum number of inodes in root filesystem。完成之后，保存并退出。准备修改内核配置：

`1`	`make kernel_menuconfig`

这里面大部分都不需要编辑，但是 OpenWrt 默认的配置没有提供对 AHCI 的支持，我们需要自己将他选中。同时多线程啊大内存什么的，自己看着改改。以上步骤准备完成了之后，开始下载需要的各种源码包，此处建议使用全局代理或者 VPN，以避免不必要的麻烦。

`1`	`make download -j100`

期间如果出现下载失败或是其他异常导致中断，你可以 make download V=s 查看详细输出，并手动下载对应的包，保存至 dl 目录下。若校验和一致，就不会再去下载了。最后就是编译啦，OpenWrt Buildroot 首先回去编译 toolchain，然后编译内核和软件包。

make

或者使用多线程加速：

`1`	`make -j24 # if you have 2 * E5 26xx`

如果过程中出现错误，同样是使用 make V=s -j1 编译，并等着看错误输出。如果想睡觉时编译呢，可以将编译日志重定向到文件中来方便查看问题：

`1`	`make V=s \| tee buid.log`

使用 UglifyJS 压缩 JavaScript 文件

Fri, 24 Feb 2017 00:00:00 +0000

UglifyJS 是一个著名的 JavaScript 源码处理工具，可以用于压缩或是格式化 JavaScript 代码。同时他也是 JavaScript 编写，所以只要有 Node.js 就能很方便的使用。其项目主页：https://github.com/mishoo/UglifyJS

安装方法很简单，假定你已经安装了 node 和 npm，你只需要执行以下命令即可完成：

`1`	`npm install -g uglify-js`

使用也是很简单的，我就举个压缩代码的例子吧，假设我们一个 script.js，需要进行压缩，可以执行以下命令：

`1`	`uglifyjs script.js -o script.min.js`

以上会对代码进行压缩，并保存为 script.min.js。值得注意的是这个压缩方法不会修改变量名，如果需要更彻底的压缩，则可以：

`1`	`uglifyjs script.js -m -o script.min.js`

实际效果的话，cnVintage 所使用的 flarum 中有一个 1.4MB 大小的 forum-7a006006.js，如果对其进行压缩能将大小缩减至 824K；如果对其使用更彻底的压缩，则能缩减至 611K，效果十分可观。

Playing with AudioContext (01)

Tue, 17 Jan 2017 00:00:00 +0000

前天尝试了一波那个什么 MediaSourceExtension，结果发现那套 API 目前限制蛮大的，而且对我来说没什么帮助（audio/x-wav 完全不正常支持，audio/mpeg 也只能在 Chrome 上使用）于是只能放弃折腾了 QAQ

昨天突然想起之前写 nanoPlayer 的时候，使用了一个叫 Audio Context 的接口，nanoPlayer 用了这个 API 里的 createAnalyzer 方法，来获得音频的频率数据，进而实现了一个频谱可视化功能。之前就注意到了这个接口中有个自定义 AudioBufferSource 的方法，可以指定若干 Float32Array 并交给浏览器播放，应该是蛮有意思的。

这里就实现一个可制定频率的正弦波音频吧，如果这个实现起来没有什么难度的话，就准备试试浏览器端解码 WAV 音频。

首先是 HTML，我们要准备一个文本框来获得指定的频率，两个按钮分别控制播放的开始与停止：

<!DOCTYPE html>
<html>
  <body>
    <input id="freq" value="440" />
    <button start>start</button>
    <button stop>stop</button>
    <script>
      // TODO: code here.
    </script>
  </body>
</html>

然后初始化若干变量：

let ctx = new AudioContext();
let frames = ctx.sampleRate;
let start = document.querySelector("button[start]");
let stop = document.querySelector("button[stop]");
let last = null;

其中 ctx.sampleRate 是 AudioContext 的采样率，这里直接将一秒钟的采样数作为帧数，产生一段持续时间为一秒的 AudioSourceBuffer 就足够了，还有 last 是用来保存上一次的 AudioSource，可以随时调用 stop 方法来终止播放。

然后就是为 start 按钮增加一个点击事件：根据文本框里的数值产生一段指定频率的音频采样：

start.onclick = (e) => {
  if (last) last.stop();

let freq = document.querySelector('#freq').value;
// 初始化一个单声道，采样率和 AudioCotnext 一致，持续时间为 1 sec 的 AudioBuffer
let audioBuffer = ctx.createBuffer(1, ctx.sampleRate, ctx.sampleRate);
// 获得其中第一个声道的数据源，类型是 Float32Array
let nowBuffering = audioBuffer.getChannelData(0);
// 填充数据，取值范围是 [-1, 1]，直接用正弦函数就行了
for (let i = 0; i < frames; ++i) {
nowBuffering[i] = Math.sin((Math.PI _ 2 _ freq) \* (i / ctx.sampleRate));
}
// 初始化一个 AudioBufferSource，并将上面的 AudioBuffer 与之绑定，并输出到 AudioContext
let source = ctx.createBufferSource();
source.loop = true;
source.buffer = audioBuffer;
source.connect(ctx.destination);
// 开始回放
source.start();
// 保存以备后续使用
last = source;
}

当然还需要对停止按钮绑定一个事件，来停止回放：

stop.onclick = (e) => {
  if (last) last.stop();
  last = null;
};

以上就是所有代码啦，效果的话，我在这里直接放个 demo 就行：

Playing with MediaSource (01)

Mon, 16 Jan 2017 00:00:00 +0000

MediaSource 是 HTML5 中的一个实验性特性，用于给 HTMLMediaElement 对象提供数据源。这里的数据源通常是使用 XMLHttpRequest 获得的数据。在 XHR 取得数据后，我们可以使用 JavaScript 对数据进行一些操作，比如提取 ID3、修改封装类型等等。某 bilibili 开源的 Flv.js 就是使用这个特性实现的在 HTML5 环境下播放 H264 + AAC 格式的 FLV 视频。

最近在写 Nano Player 的时候遇到了两个比较尴尬的问题：1、ID3 信息必须手动指定；2、网易云音乐上下载的 MP3 由于某些原因必须缓冲 2MB 才能播放（详见 MP3 缓冲 2MB 才开始播放的解决方法）。于是想到是不是可以用这个 MediaSource 来暴改 MP3 呢？~~我也不知道~~

不管怎样，要用这个 MSE 来解决问题，首先就是要用它来播放文件（如果获得的数据直接喂给 MSE 都不能播放那还讨论个什么鬼）

创建若干变量并初始化：

let mediaSource = new MediaSource();
let audio = new Audio();
let media = "audio.mp3";

audio.src = URL.createObjectURL(mediaSource);
audio.controls = "controls";

然后是对 sourceopen 事件添加监听。当 sourceopen 事件触发后，我们需要创建一个 sourceBuffer，随后使用 XMLHttpRequest 获得媒体文件的内容，并附加到这个 sourceBuffer 内。当以上工作完成后，调用 play 方法播放音频：

mediaSource.addEventListener("sourceopen", () => {
  let sourceBuffer = mediaSource.addSourceBuffer("audio/mpeg");

  let xhr = new XMLHttpRequest();
  xhr.open("GET", media);
  xhr.responseType = "arraybuffer";
  xhr.onreadystatechange = () => {
    if (xhr.readyState === xhr.DONE && xhr.status === 200) {
      sourceBuffer.addEventListener("updateend", () => {
        mediaSource.endOfStream();
        audio.play();
      });
      sourceBuffer.appendBuffer(xhr.response);
    }
  };
  xhr.send();
});

最后，为了方便检查，将 audio 添加到 body 中：

`1`	`document.querySelector("body").appendChild(audio);`

~~嗯，以上的这些代码就能实现一个本来只要三行代码就完成的工作了。~~ 具体效果可以看这里：MSE Playground 01。下面需要解决的问题是，为什么刚开始尝试使用的 M4A 格式无法正常播放 ~~，这个坑死人了我哪知道只能 MP3 啊浪费了两个多小时……~~。

由于目前来看，MediaSource 的表现在各个浏览器中并不一致，同时支持的格式就目前而言还是十分有限，这个系列估计就这一篇了吧 23333（逃

参考链接：

Simple Policy Routing on OpenWrt

Wed, 30 Nov 2016 08:00:00 +0000

上学期期末移动宽带质量不断劣化，我们宿舍被迫转向电信宽带。为了解决那个该死的防共享机制，我自费购买了一个坑爹的破解路由器，本质就是一个替换了 PPP 的 OpenWrt 路由器。选择购买而不是自己动手破解的主要原因是不想在这上面消耗太多精力，既然有现成的解决方案，那就掏钱解决了（

那个破解的路由器很渣，是个看起来就不像那种能拖宿舍里所有电子产品的破路由，所以我只将其作为拨号路由，而主要的 NAT 工作和 Access Point 仍然交给 WNDR4300 来完成。电信的路由器 LAN IP: 192.168.43.1 并开启 DHCP，然后 WNDR 4300 直接将 WAN 接至任意一个 LAN 口，完成后顺手关掉了破解路由器的 WiFi

当然大家都知道电信的国际出口有多惨，上个 Google 查东西都让人想砸电脑（不），同时宿舍移动的接口不用白不用。于是决定尝试一下组两个网关，让部分海(fan)外(qiang)流量走江苏移动，其余的走江苏电信，来提升整体的网络体验

首先是要修改 WNDR4300 上的交换机配置，让他将一个 LAN 口以 WAN 的形式工作：在 OpenWrt 的交换机配置界面，添加一个 VLAN，ID 任意，将 CPU 和准备当作 WAN 的那个口分别设置为 tagged 和 untagged，剩下的其他端口设置成关就好了。

然后去网络 -> 接口，添加一个类型为 PPPoE 的接口，物理设置接口为刚刚创建的 VLAN 接口。因为路由器的主要接口仍然是之前的 DHCP 客户端，所以我们要去掉高级设置中的使用默认网关。填写好 PAP/CHAP 用户名/密码后，提交变更。不出意外就成功的拨号完毕，下面需要的就是如何策略路由了。

使用 SSH 连接到路由器，更新软件包并安装 iproute2。完成后输入 ip route，可以看到类似这样的一行结果：

`1`	`10.49.196.1 dev pppoe-cmcc proto kernel scope link src 10.49.199.127`

很容易想到 10.49.196.1 就是移动 PPPoE 的网关了，我们只要将部分网段的默认路由修改到这个网关上，就可以实现高速访问国际互联网了（

#!/bin/sh

INT=pppoe-cmcc
TABLE=cmcc

DEST=`ip route show dev $INT | head -n 1 | awk '{print $1}'`

# ConoHa JP
ip route add 133.130.96.0/19 via $DEST
ip route add 133.130.88.0/21 via $DEST
ip route add 133.130.126.0/24 via $DEST

# Linode JP
ip route add 106.185.24.0/21 via $DEST
ip route add 106.186.16.0/20 via $DEST

# CCZU CERNET
ip route add 219.230.144.0/20 via $DEST

# JS Cernet
ip route add 211.65.74.0/24 via $DEST

# CCZU Spoc
ip route add 202.195.100.0/24 via $DEST

嗯，最基本的目的已经达到了。无压力播放 1080P 清晰度的油管视频最棒了（

不过不久又有新的问题了，江苏电信的校园宽带采用的是动态密码，PPPoE 断开后因为密码失效，无法自动恢复接口。这对于寝室一个玩梦幻西游的同学来说是非常恐怖的——每次掉线都要花大量的时间重新登陆所有的账号。

于是，经过讨论决定，将 192.168.1.240/28 这个网段献祭出来，做纯移动的接口，以用来挂机下载或者是长时间不掉线的网络。首先我们需要在 /etc/iproute2/rt_tables 里增加一行 250 cmcc 来增加一个路由表，之后使用 ip rule 将相关网段的查找规则设置成这个新表即可。

经过编辑的新脚本如下：

#!/bin/sh

INT=pppoe-cmcc
TABLE=cmcc

DEST=`ip route show dev $INT | head -n 1 | awk '{print $1}'`
SRC=`ip route show dev $INT | head -n 1 | awk '{print $7}'`

if [ "$1" == "pppoe-cmcc" ]; then # Remove all old rules # while ip rule delete from 0/0 to 0/0 table cmcc 2>/dev/null; do true; done

    # Add rules to let 192.168.1.240/28's traffic to CMCC gateway.
    ip rule add from 192.168.1.240/28 table cmcc priority 32765
    ip route add default via $SRC dev $INT table cmcc

    # ConoHa JP
    ip route add 133.130.96.0/19 via $DEST
    ip route add 133.130.88.0/21 via $DEST
    ip route add 133.130.126.0/24 via $DEST

    # Linode JP
    ip route add 106.185.24.0/21 via $DEST
    ip route add 106.186.16.0/20 via $DEST

    # CCZU CERNET
    ip route add 219.230.144.0/20 via $DEST

    # JS Cernet
    ip route add 211.65.74.0/24 via $DEST

    # CCZU Spoc
    ip route add 202.195.100.0/24 via $DEST
fi

将这个脚本放置到 /etc/ppp/ip-up.d/cmcc 中，并添加可执行权限，我们的简单的策略路由脚本就能在移动拨号成功后自动应用到系统路由表上。

Access IPv6 Network with Hurricane Electric's Tunnel Broker and PPTP

Mon, 24 Oct 2016 08:00:00 +0000

这几天一直折腾~~没啥意义的~~IPv6，首先是看到了 @蔓舞寻樱 dalao 的文章：利用代理隧道接入 HE.net 的 IPv6 网，就顺便在阿里云的学生优惠服务器上尝试部署了一下，拿到了前缀 2001:470:36:a90，前缀长度为 64 的 IPv6 地址池，就是这个速度实在是比较尴尬 (:3」∠)，从青岛阿里云到江苏教育网的 MTR 输出大致这样（╯－＿－）╯╧╧

My traceroute [v0.85]
ntzyz-aliyun (::) Sat Sep 24 14:46:35 2016
Keys: Help Display mode Restart statistics Order of fields quit
Packets Pings
Host Loss% Snt Last Avg Best Wrst StDev

1.  zhangyuze320602-1.tunnel.tserv25 0.0% 10 408.3 408.8 408.1 410.3 0.5
2.  10ge1-6.core1.sin1.he.net 0.0% 10 400.0 405.3 399.7 409.9 4.5
3.  10ge1-16.core1.hkg1.he.net 0.0% 10 432.5 438.2 432.0 464.2 9.7
4.  100ge10-1.core1.tyo1.he.net 0.0% 10 484.9 488.1 484.5 494.7 4.0
5.  10ge1-13.core1.lax2.he.net 0.0% 10 584.6 590.1 583.7 600.6 6.2
6.  cngi-bjix-as-ap-as23911.10gigabi 77.8% 10 587.2 587.2 587.2 587.3 0.0
7.  2001:252:0:302::1 0.0% 10 731.0 730.9 730.4 731.2 0.0
8.  2001:252:0:100::1 88.9% 10 729.8 729.8 729.8 729.8 0.0
9.  2001:252:0:1::1 11.1% 10 729.7 729.5 729.2 730.0 0.0
10. 2001:da8:1:1006::2 11.1% 10 736.0 734.4 731.6 739.2 2.4
11. 2001:da8:257:0:101:4:19:11 0.0% 10 727.7 738.3 727.7 776.6 16.0
12. 2001:da8:257:0:101:4:1:5 0.0% 10 727.4 727.9 727.4 728.6 0.0
13. 2001:da8:257:0:101:4:1:2 0.0% 10 736.9 736.5 732.0 739.8 2.4
14. ???
15. 2001:da8:257:0:101:4:67:e 0.0% 8 756.8 758.2 756.8 766.4 3.2
16. 2001:da8:257:0:101:4:67:1 12.5% 8 762.1 762.4 761.9 763.5 0.4
17. 2001:da8:1:504::2 0.0% 8 768.0 766.4 765.0 768.2 1.1
18. cernet2.net 12.5% 8 767.5 768.0 767.5 768.5 0.0
19. 2001:da8:1008:500::2 0.0% 7 767.0 767.2 766.7 767.8 0.0
20. 2001:da8:1008:20::1 16.7% 7 788.4 774.2 770.0 788.4 7.9
21. 2001:da8:1008:1030:bdd2:aba0:a20 0.0% 7 816.3 791.0 770.0 831.8 26.6

不过再怎么说也算是有了自己的 IPv6 地址了（这延迟这速度能用？）。但是这手里有这么多 IPv6 地址不搞点什么也真是太浪费了，所以想到的问题就是，有没有什么简单的办法，能让任意设备接入这个隧道？首先想到的是使用 sit 隧道，直接借助 6in4 实现这个功能，但是实际上这个隧道在很多时候不可用（比如防火墙不允许 6in4 流量），同时两端的 IPv4 地址必须指定，不是很灵活（虽然几个脚本就能解决但是我懒），遂放弃 sit 隧道。

再然后想到的就是虚拟专用网络了，VPN 可以通过互联网传递内网信息，因为可以通过这项技术来加密流量并改变出口，VPN 也被大范围的拿来用作科学上网的工具。此处我只是需要试验一下这个思路是否可行，所以只搭建了简单而并不算安全的 PPTP VPN。我使用的服务器安装的系统为 Ubuntu 14.04.5 LTS，其他发行版操作也是类似的。首先安装 pptpd

`1`	`sudo apt install -y pptpd`

安装完成后，修改几个配置文件，印象里要编辑的就是 /etc/ppp/chap-secrets, /etc/ppp/pptpd-options 和 /etc/pptpd.conf，这些在网上都有大量的教程，我就不赘述了（其实还是懒）。通常的安装步骤都需要编辑 /etc/sysctl.conf 来启用 IPv4 Forward 功能并编辑 iptables 的 NAT 表，来实现转发 IPv4 数据包的功能，但是我并不打算用这个隧道走 IPv4 流量，这些步骤就可以忽略不看了。

配置完成后，就可以在 Windows 下的网络和控制中心中创建一个新的 VPN 连接来测试一下，协议类型选择 PPTP，身份验证选 MS-CHAPv2。创建完成后连接，如果没有其他问题就能成功联通，获得了一个 IPv4 地址（同时你会发现你上不了网了 2333）。如果出现其他问题，可以参照 /var/log/syslog 和 Windows 下的错误代码，面向 Google 进行调试。

下面就是如何让这个 PPTP 支持 IPv6 了。首先，我们要修改一下 /etc/ppp/pptpd-options，在文件最后添加一些内容：

cat >> /etc/ppp/pptpd-options << EOF
ipparam pptpd
ipv6 ,
EOF

然后就是安装 radvd (Linux IPv6 Router Advertisement Daemon)：

`1`	`sudo apt install -y radvd`

完成后，你需要创建一个脚本，他们会在每个 ppp 连接建立之后运行，来完成 IPv6 相关的设置。首先是 /etc/ppp/ipv6-up.d/radvd，你需要将其中的 IPv6 地址（2001:470:35:a90）换成你自己的地址：

#!/bin/sh
if test $PPP_IPPRARM != pptpd ;then
        exit 0
fi

ADDR=$(echo $PPP_REMOTE | cut -d : -f 3,4,5,6)

if test x$ADDR == x ; then
        echo "Unable to generate IPv6 Address"
        exit 0
fi
ADDR=2001:470:35:a90:$ADDR

#add route
route -6 add $ADDR/128 dev $PPP_IFACE

#generate radvd config
RAP=/etc/ppp/ipv6-radvd/$PPP_IFACE
RA=$RAP.conf

cat <<EOF >$RA
interface $PPP_IFACE{
AdvManagedFlag off;
AdvOtherConfigFlag on;
AdvSendAdvert on;
MinRtrAdvInterval 5;
MaxRtrAdvInterval 100;
UnicastOnly on;
AdvSourceLLAddress on;
prefix 2001:470:35:a90::/64 {};
};
EOF

#start radvd
/usr/sbin/radvd -C $RA -p $RAP.pid

#start tchdpd
/usr/sbin/tdhcpd \n --dns-server=2001:470:20::2 \n --dns-name=$PPP_IFACE.tunnel.ipv6.icybear.net \n --pid-file=$RAP.dhcp.pid \n --local-id=tunnel.ipv6.icybear.net -L debug\n $PPP_IFACE

#update dns
ARPA=$(ipv6_rev $ADDR)
nsupdate << EOF
update delete $ARPA
update add $ARPA 10 ptr $PPP_IFACE.tunnel.ipv6.icybear.net
send
update delete $PPP_IFACE.tunnel.ipv6.icybear.net
update add $PPP_IFACE.tunnel.ipv6.icybear.net 10 aaaa $ADDR
send
EOF

exit 0

再然后是 /etc/ppp/ipv6-down.d/radvd：

#!/bin/sh

RAP=/etc/ppp/ipv6-radvd/$PPP_IFACE

kill `cat $RAP.pid` || true
kill `cat $RAP.dhcp.pid` || true

rm -f $RAP.\*

ADDR=$(echo $PPP_REMOTE | cut -d : -f 3,4,5,6)
ADDR=2001:470:35:a90:$ADDR
ARPA=$(ipv6_rev $ADDR)

nsupdate << EOF
update delete $ARPA
send
update delete $PPP_IFACE.tunnel.ipv6.icybear.net
send
EOF

exit 0

最后，给他们可执行权限并重启 pptpd：

1
2

chmod +x /etc/ppp/ipv6-*.d/radvd
systemctl restart pptpd.service

不出意外的话，再进行 PPTP 连接后，你就可以获得一个 IPv6 地址啦~~但是他真的非常慢（废话）~~

完成这个 IPv6 Over PPTP 后，我开始有些奇怪的想法：由于阿里云的 IPv6 地址池是走的 sit 隧道，速度非常慢，同时 Linode 机房可是有原生的 IPv6 支持的，所以如果先开一个 ticket 给自己的 Linode 搞到一个 V6 地址池，并把这个 PPTP 放到 Linode 上，会不会好很多呢？（望天）于是我准备试一下，但是装 PPTP 的时候出现了问题：

Setting up pptpd (1.4.0-7ubuntu0.1) ...
Job for systemd-modules-load.service failed because the control process exited with error code. See "systemctl status systemd-modules-load.service" and "journalctl -xe" for details.
dpkg: error processing package pptpd (--configure):
subprocess installed post-installation script returned error exit status 1
Errors were encountered while processing:
pptpd
E: Sub-process /usr/bin/dpkg returned an error code (1)</code>

查了一下似乎是 Ubuntu 的新 Bug（掀桌）。既然是 bug 那么想必过一段时间就会修复。其实在 Linode 上跑 PPTP 是一件十分不安全的事儿（GFW is watching you）搞不好自己的服务器就被搞了那还玩个什么鬼，所以觉得是时候用 OpenVPN 代替 PPTP，使用 OpenVPN over Shadowsocks 来获得一个 IPv6 地址，但是似乎吃力不讨好啊……

所以还是勉强接受了这个国内的 PPTP 隧道了（躺地

[HDU1008] Elevator

Fri, 23 Sep 2016 08:00:00 +0000

题目很简单，一次就能 A，把代码拉过来测试一下发表文章~

#include <iostream>

int main() {
    int n;
    while (std::cin >> n) {
        if (!n)
            return 0;
        int currFloor = 0;
        int sum = 0, aim;
        while (n--) {
            std::cin >> aim;
            sum += ((currFloor > aim) ? (4) : (-6)) * (currFloor - aim) + 5;
            currFloor = aim;
        }
        std::cout << sum << std::endl;
    }

    return 0;
}

Things around my personal Minecraft server

Sun, 21 Aug 2016 08:00:00 +0000

按照以往运行 Minecraft 服务器的情况，玩家们常常需要一些游戏以外的信息：

服务器负载情况
服务器是否还活着
我~~是死宅~~要上传皮肤

那么，作为一个不怎么管服务器里发生了什么的管理，我就主动把这些功能实现了一下。其他的游戏内管理就甩交给 @kasora 了。

服务器负载

我们目前使用的是腾讯机房的双核 4G 的云服务器，讲道理这种阵容并不适合跑 MC 这种单线程大作。与此同时这次服务器应要求添加了村庄 Mod（Millenaire），而 Millenaire 的作者自述此 Mod 和 Bukkit 存在兼容性问题，直接导致我们不再能使用 Bukkit 或者是基于 Bukkit 的服务端（例如 Cauldron），进而失去了 Bukkit API 和民间带来的一些优化。于是，在任意时刻知道服务器负荷以及一段时间内的负荷就显得比较有意义了。

至于如何去实现，就比较简单了。由于备案之类的种种问题，我们运行着 Minecraft Server 的服务器原则上不可以运行任何 HTTP Server 服务，所以我的思路是本机采集数据，然后通过某些办法发送到另一个已经备案了的服务器。

收集数据

在 Linux 设备上，我们有很多途径可以知道设备的负载和内存开销。系统负载的话，比如 top、uptime 命令，或者是 /proc/loadavg 这类文件，同样系统时间和内存占用也不难获得。于是我们可以准备这么一个 shell 脚本：

#!/bin/bash

TIMESTAMP=`date +%s`
LOAD=`cat /proc/loadavg | awk '{print $1}'`
MEM=`free -m | grep 'Mem' | awk '{print $3}'`

URL="https://status.mc.ntzyz.cn/set.php?timestamp=$TIMESTAMP&mem=$MEM&load=$LOAD"

curl $URL > /dev/null

执行一次，就能得到当前时间戳，1 分钟内负载和内存使用，并把这些数据作为请求发送到 status.mc.ntzyz.cn 所在的服务器了。这个脚本需要定时执行，使用 crontab 就可以完成。执行 crontab -e，并在文末添上：

`1`	`* * * * * /bin/bash /home/ntzyz/send.sh`

就可以实现每分钟一次请求了。

记录数据

服务器的各种数据发送到 status.mc.ntzyz.cn 之后，就需要将这些数据保存下来，准备随时被需要数据的情况下使用。是时候让全世界最好的语言上场了（雾

<?php
  if ($_SERVER["REMOTE_ADDR"] != "115.159.105.52") { // 防止某些人用自己的数据没事儿干瞎 GET
    $res->status = "ERROR";
    $res->message = "You shall not access.";
    die(json_encode($res));
  }

  $conn = mysql_connect("localhost", "_(:3」∠)_", "（╯－＿－）╯╧╧");
  if (!$conn) {
    $res->status = "ERROR";
    $res->message = "Could not connect: " . mysql_error();
    die(json_encode($res));
  }

  $mem_used = $_GET["mem"];
  $cpu_load = $_GET["load"];
  $timestamp = time();
  if (!isset($mem_used) || !isset($cpu_load)) {
    $res->status = "ERROR";
    $res->message = "required field(s) is empty.";
    die(json_encode($res));
  }

  mysql_select_db("mc_stat", $conn);
  $sql = "insert into mc_stat(timestamp, cpu_load, mem_used) values('$timestamp', '$cpu_load', '$mem_used')";

  if (!mysql_query($sql,$conn)) {
  $res->status = "ERROR";
    $res->message = "SQL Error: " . mysql_error();
    die(json_encode($res));
  }
  else {
  $res->status = "success";
    echo(json_encode($res));
  }
?>

这样你就可以将使用 cURL 发来的数据保存到数据库了。然后就是再写一段脚本来读取这些数据，准备让其他人读取。

<?php
  header("Access-Control-Allow-Origin: *"); // Ajax is GOOD
  $conn = mysql_connect("localhost", "（╯－＿－）╯╧╧", "_(:3」∠)_");
  $limit = 15;
  if (isset($_GET["limit"])) {
    $limit = $_GET["limit"];
  }

  if (!$conn) {
    $res->status = "ERROR";
    $res->message = "Could not connect: " . mysql_error();
    die(json_encode($res));
  }

  mysql_select_db("mc_stat", $conn);

  $res = mysql_query("select \* from mc_stat order by timestamp desc limit $limit");
  $i = 0;

  $resp->status = "success";
  $resp->data = array();

  while ($row = mysql_fetch_array($res)) {
  $resp->data[$i]->timestamp = $row["timestamp"];
    $resp->data[$i]->cpu_load = $row["cpu_load"];
    $resp->data[$i]->mem_used = $row["mem_used"];
  $i = $i + 1;
  }

  die(json_encode($resp));
?>

数据的保存和读取就都完成了。

显示数据

世界上没什么比图形好的数据可视化方案了。我选择使用 highcharts 这个图表框架。由于只需要简单的显示两张折线图（负载和内存），我就直接对着 Demo 改了改，也算是 Make data come alive 了。两张表的结构基本一致，所以只要准备一份模板，然后两次填入数据并使用框架就行了。首先是模板：

var template = {
  title: {
    text: "Monthly Average Temperature",
    x: -20, //center
  },
  xAxis: {
    categories: null,
  },
  yAxis: {
    plotLines: [
      {
        value: 0,
        width: 1,
        color: "#808080",
      },
    ],
  },
  tooltip: {
    valueSuffix: "",
  },
  legend: {
    layout: "vertical",
    align: "right",
    verticalAlign: "middle",
    borderWidth: 0,
  },
  series: [
    {
      name: "Tokyo",
      data: [],
    },
  ],
};

嗯，稍微了解过 Highcharts 的人都能看出，这基本就是第一个 Demo —— Basic Lines 的内容，只是少了一些个数据。然后就是使用 Ajax，将保存在 status.mc.ntzyz.cn 上的数据获得到浏览器，填入 template 并显示。

$.ajax({
  url: "https://status.mc.ntzyz.cn/get.php?limit=" + limit,
  type: "GET",
  beforeSend: function () {
    $("#loadchart").html("获取数据中");
    $("#memchart").html("获取数据中");
  },
}).done(function (data) {
  var res = JSON.parse(data);
  template.xAxis.categories = Array(limit)
    .fill(0)
    .map(function (it, off) {
      return limit - off;
    });
  template.title.text = "Load Average";
  template.series[0].name = "Load";
  template.series[0].data = Array(limit)
    .fill(0)
    .map(function (it, off) {
      return res.data[off].cpu_load * 1;
    })
    .reverse();
  $("#loadchart").highcharts(template);
  template.title.text = "Memory Used";
  template.series[0].name = "MB";
  template.series[0].data = Array(limit)
    .fill(0)
    .map(function (it, off) {
      return res.data[off].mem_used * 1;
    })
    .reverse();
  $("#memchart").highcharts(template);
});

这样，就完成了所有的数据采集、记录和显示工作了。在那段 PHP 脚本中可以看出，只要修改 limit 的值，就能获得不同数量的数据，因此短时间的数据显示和长时间的数据显示就都可以完成了。

皮肤配置、上传与预览

去年暑假剁了正版 Minecraft 之后才知道这个游戏是有皮肤这个设定的，然而在 Online Mode 设置为 Off 的服务器上，客户端并不会去从 Mojang 的服务器上获得其他玩家的皮肤数据，这样别人就看不到我的皮肤了（雾

皮肤配置

由于没有使用正版验证的玩家是不能通过官方途径更换自己的皮肤，我们需要使用额外的 Mod 来实现皮肤的自定义。这里我们使用了 UniSkinMode，配置的方法也不算麻烦。

皮肤上传

按照 UniSkinMod 的 API，我们需要准备每个玩家的 JSON 信息，同时准备一个 textures 目录来存储 PNG 格式的皮肤。上传皮肤只需要修改对应的 JSON 文件，同时添加/更新 textures 目录下对应的文件即可。综上，这些工作只需要一段 PHP 就可以完成。

<?php
  header("Access-Control-Allow-Origin: \*");
  if (!isset($\_GET['user']))
    die('something wrong');

  $user = $_GET['user'];
  $target_dir = "/var/www/skin/textures/";
  $target_file = $target_dir . $_GET['user'];
  $imageFileType = pathinfo($target_file,PATHINFO_EXTENSION);

  if (!move_uploaded_file($_FILES["file"]["tmp_name"], $target_file)) {
    if (!$\_FILES["file"])
    die('file not set');
    die('上传失败');
  }
  else {
    $file = fopen("/var/www/skin/" . $user . ".json", "w");
      fwrite($file, "{"player_name": "". $user ."",  "last_update": 1416300800,  "model_preference": ["default"],  "skins": {    "default": "". $user .""  }}");
      fclose($file);
    die('上传成功');
  }

皮肤预览

做这个纯粹是我比较想折腾折腾，毕竟 WebGL 出来这么多年了，同时也有 Three.js 这种很好用的 JavaScript 3D 库。这里的工作相当繁杂（然而逻辑很简单，都特么 UV 贴图数据要手撕）。想了解的可以直接去看下源码。

AS142641

Mon, 01 Jan 0001 00:00:00 +0000

An IPv6 only autonomous system operated by @ntzyz.

HE BGP Toolkit: AS142641 Yuze Zhang

Peering policy

AS142641 has an open peering policy, we are willing to peer over any exchange points we have in common, or through tunnels.

Contract peering [at] as142641.net for details.

(Abuse) contact

noc [at] as142641.net, or t.me/ntzyz.

CoreMark on WebAssembly

Mon, 01 Jan 0001 00:00:00 +0000

Sort by：

Filter:

Browser	Device	OS	Arch	CPU	CM	Submit by
Safari/605.1.15	MacBook Pro (16-inch, 2019)	macOS 11.1	x86_64	i7-9750H	19957.424162	@ntzyz
Chrome/87.0.4280.88	MacBook Pro (16-inch, 2019)	macOS 11.1	x86_64	i7-9750H	17132.088402	@ntzyz
Firefox/84.0	MacBook Pro (16-inch, 2019)	macOS 11.1	x86_64	i7-9750H	16153.784024	@ntzyz
Safari/604.1	iPhone 12 mini	iOS 14.3	aarch64	Apple A14	19229.536568	@ntzyz
Safari/604.1	iPhone 11 Pro	iOS 14.3	aarch64	Apple A13	16633.399867	@ntzyz
Safari/605.1.15	MacBook Pro (13-inch, M1, 2020)	macOS 11.1	aarch64	Apple M1	23432.008123	@Hanataba
Safari/605.1.15	iPad 2020	iPadOS 14.3	aarch64	Apple A12	15384.615385	@ShellBin
Chrome/87.0.4280.88	Lenovo Legion R7000	Windows 10 20H2	x86_64	R7-4800H	19088.826673	@ShellBin
Chrome/81.0.4044.117	Redmi K30 5G	Android 11 (MIUI 12.1)	aarch64	Snapdragon 765G	8131.283264	@ShellBin
Edge/87.0.664.66	-	Windows 10 2004	x86_64	R9-3900X	19945.482348	@Hatsuroku
Safari/604.1	iPhone 5S	iOS 12.4.8	aarch64	Apple A7	4357.298474	@ShellBin
Safari/605.1.15	X570-A PRO	macOS 11.1	x86_64	R7-3700X	21739.130435	@ntzyz
Safari/605.1.15	iPhone 7	iOS 14.2	aarch64	Apple A10	11397.959765	@ZephRay
Safari/605.1.15	iPad Air 2	iPad OS 14.2	aarch64	Apple A8X	6764.651620	@ZephRay
Edge/87.0.664.66	Dell Mobile Precision 5540	Windows 10 1809	x86_64	Intel Core i7-9850H	17446.932248	@ZephRay
Safari/605.1.15	iPhone 6	iOS 12.4.9	aarch64	Apple A8	5882.982137	@ZephRay
Safari/605.1.15	MacBook Air (13-inch, M1, 2020)	macOS 11.1	aarch64	Apple M1	23466.833542	@ZephRay
Safari/604.1	iPad Pro 2020	iOS 14.3	aarch64	Apple A12Z	15128.593041	@ntzyz
Edge/87.0.664.66	Lenovo ThinkPad X250	Windows 10 2004	x86_64	i5-4300U	9837.238419	@ZephRay
Chrome/87.0.4280.101	Samsumg SM-P610	Android 10	aarch64	Exynos 9611	4548.555834	@ZephRay
Safari/605.1.15	MacBook Pro (15-inch, Mid 2015)	macOS 11.1	x86_64	Core i7-4870HQ	16684.741804	@ZephRay
Safari/-	iPod Touch 6th	iOS 12.?	aarch64	Apple A8	4740.459824	@ntzyz
Edge/87.0.664.66	Samsung Q1 Ultra	Windows 7 SP1	aarch64	Intel A110	896.422459	@ZephRay
Chrome/72.0.3626.121	Pixel C	Android 9	aarch64	Tegra X1	3553.239370	@spinmry
Chrome/87.0.4280.101	Xperia 1 II	Android 10	aarch64	Snapdragon 865	10031.004924	@Ethy1eneL
Chrome/87.0.4280.101	OnePlus 8	Android 11	aarch64	Snapdragon 865	10365.916865	@paizhang
Firefox/84.1.2	Honor Play	Android 9	aarch64	Kirin 970	3559.141061	@AlexGuo
Safari/? (12.1.2, 14607.3.9)	MacBook Pro 15" Late 2013	Mac OS X 10.14.6	x86_64	Core i7-4960HQ	16112.140498	@paizhang
Edge/?	Samsung Galaxy Tab S6	Android 10	aarch64	Snapdragon 855	10671	@AlexGuo
Chrome/87.0.4280.88	-	Windows 10	x86_64	Ryzen 5 3600	15578.750584	@Ethy1eneL
Chrome/71.0.3578.99	HTC Droid DNA	Android 4.1.1	armhf	Snapdragon APQ8064	1917.913310	@ZephRay
Chrome/87.0.4280.88	ThinkPad X230	Windows 7 SP1	x86_64	i5-3320M	9301.460329	@paizhang
Edge/?	-	Windows Server 2019	x86_64	Intel Xeon E5-2420v2	10025.519504	@spinmry
Chrome/87.0.4280.88	ThinkStation D30	Windows 10	x86_64	E5-2696v2	10072.337698	@Ethy1eneL
Chrome/81.0.4044.117	Sony Xperia P	Android 5.0.2	armhf	STE NovaThor U8500	781.971991	@Ethy1eneL
Safari/605.1.15	MacBook Air 13" 2017	macOS 11.1	x86_64	i5-5350U	14048.890138	@ZephRay
Chrome/87.0.4280.141	-	Windows 10	x86_64	TR-3970X	20140.980906	@haruue
Chrome/87.0.4280.141	Xiaomi MI11	Android 11	aarch64	Snapdragon 888	10040.160643	@UiharuNeko
Safari/605.1.15	Mac mini (M1, 2020)	macOS 11.2.3	aarch64	Apple M1	23360.847220	@ntzyz
Firefox/88.0	Mac mini (M1, 2020)	macOS 11.2.3	aarch64	Apple M1	12945.821736	@ntzyz
Chrome/90.0.4430.85	Mac mini (M1, 2020)	macOS 11.2.3	aarch64	Apple M1	21454.623471	@ntzyz
Safari/604.1	iPhone 12 Pro Max	iOS 14.5	aarch64	Apple A14	19125.334693	@ntzyz
Firefox/68.0 (龙芯浏览器2.9)	Lenovo Unknown	Fedora 28; Linux 5.4.60	mips64le	LS3A4000	3418.219108	@spinmry
Edge/83.0.478.50	ThinkPad X61	Windows 7 SP1	x86_64	Intel Core 2 Duo P9700	8536.396089	@ZephRay
Safari/605.1.15	MacBook Pro (14-inch, M1 Pro, 2021)	macOS 12.0.1	aarch64	Apple M1 Pro	23629.489603	@ntzyz
Safari/605.1.15	MacBook Pro (16-inch, M1 Max, 2021)	macOS 13.0 (Beta)	aarch64	Apple M1 Max	24964.633436	@ntzyz
Edge/103.0.1264.44	-	Windows 10 20H2	x86_64	Core i9 12900KF	34498.620055	@张建新
Safari/605.1.15	MacBook Pro (14-inch, M1 Pro, 2021)	macOS	aarch64	Apple M1 Pro	24762.690879	@张建新
Safari/605.1.15	iPad Pro (12.9-inch, M2, 2022)	iPadOS 16.1	aarch64	Apple M2	22824.102252	@ntzyz
Safari/605.1.15	iPhone 14 Pro Max	iOS 16.1	aarch64	Apple A16	24614.374795	@ntzyz

* Your browser:
Compile command: make PORT=linux CC=emcc EXE=.html
Try it with your own browser here: https://ntzyz.io/cm/

b) { return 1; } return 0; }) document.querySelector('tbody').innerHTML = '' + data.map(function (el) { return el.html }).join('') + '' window.filter() }; window.filter = function () { var filter = document.querySelector('input').value document.querySelectorAll('table#cm tbody tr').forEach(function (el) { if (el.innerText.toLowerCase().indexOf(filter.toLowerCase()) < 0) { el.style.display = 'none'; return } el.style.display = ''; }) } document.querySelector('#UA').innerText = navigator.userAgent" style="width: 0; height: 0" />

Friends

Mon, 01 Jan 0001 00:00:00 +0000

Friends

Mon, 01 Jan 0001 00:00:00 +0000

My Friends.

Links

Mon, 01 Jan 0001 00:00:00 +0000

Some useful link to reach me at other platforms.

Search

Mon, 01 Jan 0001 00:00:00 +0000

Yuze Network | AS142641

Mon, 01 Jan 0001 00:00:00 +0000

An IPv6 only autonomous system operated by @ntzyz.

HE BGP Toolkit: AS142641 Yuze Zhang

Peering policy

AS142641 has an open peering policy, we are willing to peer over any exchange points we have in common, or through tunnels.

Contract peering [at] as142641.net for details.

(Abuse) contact

noc [at] as142641.net, or t.me/ntzyz.

ntzyz's space

Early Experience with ZFS RAIDZ Expansion

Compiling and Installing OpenZFS 2.3.0

Upgrading a Zpool to Use RAIDZ Expansion

Adding Disks to a Zpool

Some Minor Drawbacks of Expansion

ZFS Expansion: turn a mirror in to a RAID

Run Ray-Tracing in Browser with WebAssembly

Installing Emscripten

Getting and Modifying the Source Code of “Ray Tracing in One Weekend”

Writing JavaScript Glue Code

Try It Out Yourself

Play Helldivers 2 remotely with Moonlight and USB/IP project

Use SmartDNS as DNS Server on RouterOS

Enable Container mode

Prepare Network Interfaces

Create containers and configure services.

Using NETMAP to obtain persistent IPv6 addresses and control source address

Use Nix as Package Manager in macOS

Why Not Use Brew

Nix and NixOS

“Trigger” for the Replacement

Installing nix

Installing nix-darwin

Optimizing multi-wan internet access using BGP full table on RouterOS.

Basic network configuration

Obtain realtime BGP full routing table

Establish a tunnel between the BGP feeder and the router.

Prepare the routing filters

Bring full-table home

Hands on MikroTik RB4011iGS+RM

Hardware requirements

Software Configuration

Upstream Network Configuration

WireGuard Configuration

BGP session for intranet

Bring BGP full-table back to home

IPv6 Prefix Address Translation

Multiple VLANs and Acceleration of Game/Steam downloads

Some Other Issues Occurred

My NAS and Data Backup

Site migration

Why migration

What’s new

What has been changed

What about comments

Search Telegram History with Elastic Stack

Load NVMe Driver in UEFI Shell

Collect Server Network Traffic with Multiple IPs on one same interface

Archives

Linux Desktop on Dual Monitor with Different DPI

About

Solution for Internet Connection Sharing Broken after Reboot on Windows 10

参考资料

Migrate to Walkman From NetEase Cloud Music

Decode and playback MP3 in web browser using Web Assembly

Create Linux Container with systemd-nspawn

Extend Storage with iSCSI and USB Gadget for Your PS4

Fix HiDPI issue for Netease Cloud Music

Implementing a Web Terminal

Dynamically Evaluate JavaScript on Single-Page-Application

Implementing AES-128-ECB (School Homework)

Compiling My Own OpenWrt (x86_64)

使用 UglifyJS 压缩 JavaScript 文件

Playing with AudioContext (01)

Playing with MediaSource (01)

参考链接：

Simple Policy Routing on OpenWrt

Access IPv6 Network with Hurricane Electric's Tunnel Broker and PPTP

[HDU1008] Elevator

Things around my personal Minecraft server

服务器负载

收集数据

记录数据

显示数据

皮肤配置、上传与预览

皮肤配置

皮肤上传

皮肤预览

AS142641