Pylint magic

I am using pylint on most of my code to make sure nothing bad or too complicated sneaks in. So far it has helped me a lot in discovering stuff I would not notice, and if you are not using #pylint: disable=... too often it forces you to write better code.

I just stumbled across this:

I want to find the first element in a list of elements that is bigger than the specified one. For simplicity I am just using numbers here…

./some.py:

def function(current, *selection):
    for number in [num for num in sorted(selection) if num > current]:
        return number

But my tests show me that something bad is happening…

./test_some.py:

from some import function

NUMBERS = [5, 4, 3, 2, 1]


def test_function():
    assert function(3, *NUMBERS) == 4
    assert function(3) is None

Running the tests shows that the last assert statement goes over the for loop end exits the function, returning None.

$ pytest -vv  test_some.py
================= test session starts =================
collected 1 item

test_some.py::test_function PASSED              [100%]

================== 1 passed in 0.02s ==================

But running the tests yields no errors:

$ pylint --disable "C0114" --disable "C0116" some.py test_some.py

------------------------------------
Your code has been rated at 10.00/10

I would have expected this typical message:

Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)

Weird.

So, digging a little further, that example can be trimmed down to this:

def fun():
    for _ in []:
        return True

It seems, that the check for inconsistent-return-statements gets confused by the return inside the for loop. It does (or cannot?) not check if the input of the for loop is empty or not.

So, now I am aware of this. Anyway, I only discovered this while doing test driven development. For a moment I was confused why some assert bailed out with some None value for some edge-case that was not implemented yet..

So long!

Fixed!

Again, there has been a long time since the last post.

Somehow I lost motivation, because syntax highlighting did not really work. Black text on a very dark background is not much fun to read.

In the previous post I thought this needs to be fixed inside Firefox. But no, it seems to be intended behaviour, because at some point Chrome also stopped working. And with that, all the Electron apps suffered from the same issue.

Things went quickly then, as the bug trackers filled up with random Electron app users complaining.

So the colored SVG glyphs of the font were dropped and a new release was published. Great. Who would have thought that Electron is good for anything after all? 🤭

But the node package was not updated.

Sad, so my blog theme still was broken. But tonight it struck me: “Hey, isn’t it possible to install node packages directly from the git repos?!”

Yes it is. I tried that, and guess what:

1
2
3
4

from sys import stdout

stdout.write('It works!!1!\n')
stdout.flush()

This is great. Let’s see, if I can find more time to write about random stuff in the future…

So long!

Can't fix

So after many many debugging sessions I finally found the real culprit for this issue here.. It has nothing to do with CSS, but with SVG. Who would have guessed?!

I figured — it has something to do with the web-font, so there is this issue and also that issue. But in reality it is some regression in Firefox.

Oh no. I have spent so many hours trying to figure out what’s going on. And now I am very sad to not really be able to fix this. My knowledge of SVG font rendering inside Firefox is very very limited, so I don’t feel like solving this issue anytime soon. Sorry about that.

Reading through the original ticket some high profile sites are also affected.. Hopefully this speeds up things..

Otherwise - this is something very frustrating - not be able to fix some bug. Let’s see what the future will bring..

So long!

Weird CSS

I am puzzled. My work laptop runs on macOS Catalina. Somehow it tricked me into updating, and here we are.

After some pain in the beginning, now I can say I don’t have any issues with it.

But I noticed something odd. Only on Catalina and only on Firefox, the syntax highlighting on this Website does not work.

If I use any other Browser it is fine, and if I use Firefox on any other system everything is good.

This is weird, and I do not know why this happens, and cannot remember when this started.

Guess I’ll have some debugging to do on the long weekend.

So long!

Indentation

So, while digging around in some external library code I stumbled across this method:

def format(self, value):
    if isinstance(value, set):
        value = list(value)

    return [
        self.container.output(idx,
            val if (isinstance(val, dict)
                    or (self.container.attribute
                        and hasattr(val, self.container.attribute)))
                    and not isinstance(self.container, Nested)
                    and not type(self.container) is Raw
                else value)
        for idx, val in enumerate(value)
    ]

You can see it is very difficult to understand what is going on there.. Please do not try to understand what the method is doing or where it comes from. That’s not the point I am trying to make here - just look inside self.container.output.

The method self.container.output gets the parameter idx and either val or value based on some conditions. These conditions are the cause why it is hard to understand. It is val if three conditions are met, but the first one contains further nested conditions where one of them is again nested… Complicated.

So I took the liberty to reformat the code without changing the logic of it (just whitespace changes). It turned out to this:

def format(self, value):
    if isinstance(value, set):
        value = list(value)

    return [
        self.container.output(
            idx,
            val
            if (
                isinstance(val, dict)
                or (
                    self.container.attribute
                    and hasattr(val, self.container.attribute)
                )
            )
            and not isinstance(self.container, Nested)
            and not type(self.container) is Raw
            else value
        )
        for idx, val in enumerate(value)
    ]

Well, yes, it became much longer, but on the pro side my PEP8 linter in the editor won’t complain about bad-continuation anymore. (It still complains about the Redefining built-in ‘format’, but anyway…)

Basically I removed all the hanging indents - now it is much easier to understand how these conditions interact. That single or moved one indentation step further to the right, so one can clearly see, it has nothing to do with the ands on the outside.

Then I remembered some talk I watched years ago - If your code is easy to grasp visually, the chance someone else understands it is way higher.

Ok, let’s give it a try - how about some dirty script?

#!/usr/bin/env python3

from sys import stdin, stdout
from string import whitespace

if __name__ == '__main__':
    for line in stdin:
        stdout.write(''.join(
            char if char in whitespace else '#' for char in line
        ))

    stdout.flush()

It replaces everything but whitespace with an # character.

The first example turns out like this:

### ############ #######
    ## ################# #####
        ##### # ###########

    ###### #
        ##########################
            ### ## ################ #####
                    ## #########################
                        ### ############ ###########################
                    ### ### ########################## #######
                    ### ### #################### ## ###
                #### ######
        ### #### ### ## ################
    #

And the second one like this:

### ############ #######
    ## ################# #####
        ##### # ###########

    ###### #
        ######################
            ####
            ###
            ## #
                ############### #####
                ## #
                    ########################
                    ### ############ #########################
                #
            #
            ### ### ########################## #######
            ### ### #################### ## ###
            #### #####
        #
        ### #### ### ## ################
    #

Just by looking at these outputs without comparing it to the original code (don’t scroll up) it is suddenly possible to see the different nesting levels, and how they act together.

In my opinion, the second one shows it more clearly - although it takes up much more horizontal space. But I think this is worth it - scrolling through text on the screen takes much less effort than reading the same code over and over again…

Think of it next time you are trying to write very dense code!

So long!

Updates

So, here are some updates. I am still alive, and I have not forgotten about that website.

Originally I had the plan to write something new each month. It worked well in the beginning, but then stuff happened.

Primarily: I have a new job!

It’s a very good one, the company is pretty good, and I pretty much enjoy it there. Part of the company culture is the willingness to learn something new, so there are many places where I can bring in my knowledge. On the other hand my head is exploding, because there is so much new stuff for me to learn. I have the feeling I am improving my skills every week.

Very good.

Along with this new job came the relocation to Leipzig – finally some decent city, it’s not that big, but big enough. There are a lot of very interesting people and every day I am somewhere else to discover new stuff to do. It is a lot of fun!

That is the reason I rarely find time to take care of this website. I am very sorry. But slowly my life sorts itself out, so there will be more time in the future for personal stuff like this. Maybe I can come back to former glory and write something new on a monthly basis…

In other news: The development of the Slick Hugo theme continues. Not that fast as it used to be, primarily because I don’t have so much time to do so, but also because it slowly gets to the point where it is finished.

Worth mentioning is maybe the switch to TypeScript for all the client side logic, introduction of Hugo Pipes for CSS and JavaScript delivery for fingerprinting and of course some adjustments to keep up with recent Hugo versions.

Also – Today I killed the 447 days of uptime of the server this page is running on. I think it is a quite impressive number and also a little bit scary..

It is pretty good that some system can run stable for such a long time (have I mentioned that I love FreeBSD?!?) but very scary that all the time no kernel patches were applied…

Well, now it’s over, both userland and kernel are now fully updated and patched and everything seems to run fine again.

I discovered some issues while rebooting - It is somehow not possible to send DNS requests from within newly started jails, because my firewall is not aware of the network interface of the jail. So the webserver inside the jail cannot apply OCSP stapling and waits endlessly… Sad.

This is something I have to dig into in near future, so rebooting (and thus updating the kernel) won’t be so much pain anymore.

Maybe this is stuff for the next entry here, we will see..

So long!

Weird JavaScript

This is something I stumbled across a while ago. Someone on the internet said he was asked this question during an job interview:

Can you think of a situation where this is true:

a == 1 && a == 2 && a == 3

Well, yes.

You have to ask the question, what a is.

Is it a Number? Obviously not.

The solution is quite simple if you know how JavaScript works.

You can define a to be an Object and overwrite .valueOf:

const a = {
    i: 1,
    valueOf: function() {
        return this.i++;
    }
};

if (a == 1 && a == 2 && a == 3) {
    console.log('yeah');
}

I think this is very awkward!

Or like someone else on the internet said:

Sometimes when I’m writing Javascript I want to throw up my hands and say “this is bullshit!” but I can never remember what “this” refers to.

So long!

Generate htpasswd entries with bcrypt

This week I was digging around in the configuration of my webserver.

For some internal pages I am using Basic Auth. Looking at the password hashes I noticed they start with $apr1$.

What is that? The Apache documentation about password formats tells us that this is some kind of MD5 hash.

Thanks, but no thanks (It’s a mystery to me why I decided to use this..).

So let’s strengthen security by using blowfish!

It is possible to create bcrypt hashes using the htpasswd command. This is some utility that ships alongside Apache. I am no Apache user, and I am not willing to install it just for one small task. So what do we do?

There are some alternative implementations of just htpasswd (htpasswd.py in python and Apache::Htpasswd in perl) but they don’t support blowfish as far I can see. Sad. Alternatively, there are a lot of online generators out there. But someone must be totally crazy to use them…

Python to the rescue! If you have the bcrypt package installed, you can use this script to generate the hash:

#!/usr/bin/env python3

import bcrypt

print(
    bcrypt.hashpw(
        input().encode(),
        bcrypt.gensalt(rounds=10)
    ).decode()
)

I am using input() here, so the password won’t show up in plaintext inside the ~/.python_history file when using the interactive prompt.

If you save the script inside some file, you may use it like this to append a new entry:

echo "user:$(./htspass.py)" >> .htpasswd

So long.

Fnord Day

Happy fnord day!

23.5