Kategorie: Environment

  • JVM Classloading

    Java Byte-Code

    When you compile java-code (.java files and resources), the compiler will generate so-called byte-code (.class files, or packaged .jar files). This byte-code can’t be executed by the local operating system, but only by the Java Runtime Environment (JRE). The JRE is the environment where a Java Virtual Machine (JVM) runs. The benefits are that the byte-code can run on any machine where a JRE is present (providing you haven’t used any operating-system specific code). In addition, the Java application runs in a safe space (sandbox) without having much chance to attack the underlying OS. Another advantage can be that the byte-code is executed by a just-in-time compiler (JIT compiler). A JIT compiler compiles byte-code into machine-code that can be read and executed by the underlying OS on-the-fly, often with using profiling information to optimise the machine-code.

    public class HelloWorld {
        public static void main(String[] args) {
            System.out.println("Hello World.");
        }
    }
    

    You can see the byte by using

    javap -c HelloWorld.class

    Hierarchy of Classloaders

    Classloader Hierarchy
    Classloader Hierarchy

    A Java program usually consists of many classes that are linked to each other. These classes are loaded and initialised when they are needed during runtime. Java provides a hierarchy of different ClassLoaders that are responsible for this job

    • The Bootstrap Classloader (aka Primordial Classloader) loads the classes of the JRE, classes that can be found in $JAVAHOME/jre/lib/rt.jar. The Bootstrap Classloader is the root of all other the other Classloaders. Most of the classes are implemented in C.
    • The Extension Classloader loads classes from Extension API under $JAVAHOME/jre/lib/ext (or in the directories defined in the System-Property java.ext.dirs). The Extension Classloader is a sub-class of the Bootstrap Classloader. It is implemented in Java.
    • The Application Classloader (aka System Classloader) is the default Classloader and loads classes from the classpath (java -cp or java -classpath) or Class-Path attribute of a META-INF/MANIFEST.MF file inside of a jar-file. The System Classloader is also implemented in Java.
    • In addition, Custom Classloader can be written that can load classes from any location. These User-Defined Classloaders are derived from java.lang.ClassLoader. Custom Classloader are organised as a tree, each Custom Classloader has a parent, the System Classloader is the root Classloader for all Custom Classloaders

    Finding and Loading a Class

    Class Loading Process
    Class Loading Process

    The loading of classes happens hierarchically. To load a class, the loadClass() method of the Application Classloader is called (1).

    1.  If the Application ClassLoader has already loaded the class it simply returns it. Otherwise, the Application Classloader calls loadClass() on the Extension Classloader (2).
    2. If the Extension Classloader has loaded the class already it returns it. Otherwise, the Extension Classloader calls loadClass() on the Bootstrap Classloader (3).
    3. If the Bootstrap Classloader has loaded the class already it returns it. Otherwise, the Bootstrap Classloader calls loadClass() (4) which calls findClass() (5), which then tries to find the class in $JAVAHOME/jre/lib/rt.jar, caches (stores) and returns it (6). If the Bootstrap Classloader cannot find the class, it delegates it back to the Extension Classloader (6).
    4. The loadClass() method  (7) of the Extension Classloader then calls findClass() (8), which in turn tries to find the class in $JAVAHOME/jre/lib/ext (or in the directories defined in the System-Property java.ext.dirs), caches (stores) and returns it (9). If the Extension Classloader cannot find the class, it delegates it back to the Application Classloader (9).
    5. The loadClass() method (10) of the Application Classloader then calls findClass() (11), which in turn tries to find the class in the application classpath (java -cp or java -classpath), caches (stores) and returns it (12). If the Application Classloader cannot find the class, it throws a java.lang.ClassNotFoundException to the caller (12).

    This principle is also called delegation principle. A child Classloader can see all classes loaded by the parent Classloader (not vice-versa). The parent Classloader cannot see the classes loaded by the child Classloader (visibility principle). Each Classloader contains (loads, caches, stores) the classes it is responsible for (uniqueness principle).

    This is a recursive process. To sum it up

    1. if the Classloader has already loaded the class and returns the class from its cache,
    2. if not it asks their Parent ClassLoader
    3. if the Parent ClassLoader doesn’t have the class, the ClassLoader loads it from the associated location

    You can see how the Classloading mechanism works when you start an application with

    java -verbose:class HelloWorld

    Class Loading

    A class is loaded as soon as it is referenced by another class or until the class is initialised (lazy loading). The loading occurs by the following sequence:

    1. The JRE doesn’t know the source of the byte-code. Hence, a couple of checks are performed on the byte-code to ensure it won’t cause any harm when executed (Verification of class Files in the JVM Specification) (verify)
    2. If the byte-code passes the test, the data structures are prepared that represents attributes, methods and interfaces (prepare)
    3. Subsequently, the references are resolved (interfaces, super-classes, field-references, references in method-signatures, and local variables in methods) (resolve)

    Class Initialisation

    A class, or more precise, the static members of this class, are initialised when

    • an instance of the class is created
    • a static method of the class is invoked
    • a static field of the class is assigned
    • a static field of a class is used (and not a constant)
    • a class is accessed via reflection

    Classes are initialised in the following order

    1. a field declared in a super-class is initialised before a field in a derived class
    2. a super-class is initialised before a derived class, with the exception of super interfaces
    3. if a class initialisation is triggered by access to a static field, the corresponding class is initialised (without initialising the super-class)
    4. static fields are initialised before instance attributes
    5. instance attributes are initialised when the constructor is called. A constructor of a sub-class calls the identical constructor in the super-class first.

    You can experiment this behaviour with some classes like this

    public class SuperClass {
    
        public static int A = 200;
    
        static {
            System.out.println ("Static part of SuperClass.");
        }
    
        {
            System.out.println("Non-Static part of SuperClass is initialized");
        }
    }
    
    public class SubClass extends SuperClass {
    
        public static int A = 100;
    
        static {
            System.out.println ("Static part of SubClass.");
        }
        
        {
            System.out.println("Non-Static part of SubClass is initialized");
        }
    }
    public class ClassHierarchyDemo {
    
        public static void main (String[] args) {
            System.out.println (SubClass.A);
            SubClass subClass = new SubClass();
        }
    }

    You can have a further look at when the classes are loaded with

    java -verbose:class ClassHierarchyDemo

    Remarks

    You might not need to know how classes are loaded into the JVM when you are writing simple Java programs. However, this knowledge will become useful, when writing large applications that use different libraries, and you e.g. experience version conflicts or wondering why classes and especially members are not initialised in the way you would expect. I recommend experimenting a bit with the examples above. More information can be found in the Java Language and Virtual Machine Specifications.

     

  • SSH Protocol and Key Generation

    SSH

     is a protocol used to transfer data between to entities (client and server or services) using encryption. It's basically a

    telnet

     where data is transferred over a secure channel. This means that all data transmitted in both directions is secure from eavesdropping. In software development it is used to securely connect to a remote git repository, a test- and production server etc. A SSH client should be already installed on a UNIX systems (such as Linux or MacOSX).

    The SSH protocol

    You can use SSH to log into the remote server, transfer files and execute commands. Before you do this, you have generate a key pair on your local computer. SSH uses asymmetrical cryptography (or public-key cryptography). The so called private key (or secret key) is used for encrypting the data, while the public key is used for decryption and vice versa. Hence this key-pair is generated from the same random data. Asymmetrical algorithms are not very fast, compared to symmetrical cryptographic algorithms. Symmetrical algorithms use a single key that is shared between both sides (client and server) to encrypt the transmitted data. Hence, SSH uses public-key cryptography only for authentification (to prove that the client is allowed to access the server) and to exchange a new key (session key) that is used to encrypt and decrypt connection between client and server. Here symmetrical cryptographic is being used. The secret key is stored securely on your computer and should be never ever given to anyone else. The public key is be distributed publicly, e.g. to the remote server you want to access.

    Simplified SSH protocol for Client Authentification

    This image shows a simplified version of the SSH protocol with client authentification (usually client and server agree on the encryption algorithms and other parameter, Diffie-Hellman is used for session key generation and exchange and more). However, this simplification should be sufficient to understand the basics you need to know as a software developer.

    Generating SSH Keys

    First check if you haven’t already generated a SSH key-pair by looking into the .ssh folder in your home directory

    ls ~/.ssh

    If the directory is not present or doesn’t contain something like: id_rsa, id_rsa.pub you need to generate a new key-pair by using

    ssh-keygen

    You are being ask for the key-store directory and a passphrase (a password which can also contain spaces). I did not use any special algorithms here, so in my case RSA with a key-length of 2048 bit was used (which is considered to be a sufficient key-length for a strong security). In the image below you might notice the SHA-256 and fingerprintSHA256 is a so called hash-function, A hash-function is a one-way function which calculates a fixed-length value (hash-value) out of a arbitrary sized data. One-way means that you can’t re-create the original message from the hash-value. The result is often called a fingerprint (a compressed value of my key in this case).

    SSH Key Generation

    If you want to use other cryptographic algorithms, you can do so by using the -t option, e.g. if you want to use DSA

    ssh-keygen -t dsa

    to change the key-length, use the -b parameter

    ssh-keygen -t rsa -b 4096

    The keys (id_rsa, id_rsa.pub) should now be visible in the ~/.ssh directory. The secret key (id_rsa) is protected by the pass-phrase. You can upload the public key (id_rsa.pub) to the remote server or service you want to connect with, e.g. a git-Repository. If you have root access to the remote server you have to store the public key in.~/.ssh/authorized_keys.

    You can also let ssh-keygen in a script without the need of a pass-phrase

    ssh-keygen -f $HOME/.ssh/id_rsa -t rsa -N ''

     

     

  • iTerm2 and zsh

    This article outlines how to install my preferred development environment using iTerm2 and zsh (Z shell) together with some plugins on MacOSX. It doesn’t go into much detail and describes a hands-on installation of these tools.

    (mehr …)