30th Monthly Technical Session (MTS) Report

30th Monthly Technical Session (MTS) was held on January 27th. MTS is a knowledge sharing event, in which HDE members present some topics and have QA sessions, both in English.


The moderator of the 30th MTS was Jonas.


The first topic was ‘Introducing Yahoo! Pulsar: a Distributed Pub-Sub Messaging System’ by Okubo-san. Pub-Sub stands for publish-subscribe, a messaging pattern in software architecture. In this pattern, publishers don’t send messages directly to subscribers. According to Okubo-san, communication in pub-sub happens by utilising topics. Publishers send messages of certain topics, and subscribers receives messages of certain topics.

Pulsar is a distributed pub-sub messaging platform which is developed by Yahoo!. They have been using Pulsar since 2015, and recently the platform was made open source. Pulsar offers scalability, low latency, strong ordering and consistency guarantees, cloud service oriented design, and geo replication. Okubo-san continued by explaining Pulsar’s architecture and comparing it to other messaging platforms.


The second topic was a re-explanation of an internal service, by Matsuura-san. This time, the focus was how he solved a particular problem. In this service, there are AWS Lambda functions which access EC2 instances. Initially, Fabric was utilised by the Lambda functions to execute shell commands in the EC2 instances. However, using Fabric caused some problems, such as KeyboardInterrupt or Operation not permitted.

To solve this, he used Paramiko instead of Fabric. Using Paramiko is a bit more complicated than using Fabric though, as he needed to compile it on Amazon Linux, then include it in a Lambda function’s deployment package.


The third topic was an in-depth explanation of an HDE service, by Ogawa-san. He began by explaining one of the backend components of the service. Then, he continued by teaching us how he solved a problem regarding filename encoding of ZIP file format. According to Ogawa-san, we can do this in two ways. The first one is to utilise language encoding flag (general purpose bit 11). The second one is to utilise Info-ZIP’s unicode path extra field.


The fourth topic was an explanation of an HDE service’s new feature, by Iskandar-san. He began by briefly reintroducing the service itself. Then, he immediately proceeded to teach us everything about the new feature, such as its concept, its architecture, its development process, and other technical details. He also talked about the changes the new feature will bring, such as additions to the user interface and new use cases. He had even made a video which explains how the new feature will actually work once it is released.


The fifth topic was about Bitcoin and blockchain), by Kelvin-san. He is our current Global Internship Program (GIP) participant. He began by explaining what Bitcoin is, particularly by comparing it to the existing financial system. Then, he taught us blockchain, a database that serves as Bitcoin’s ledger. Blockchain is a distributed database maintaining a continuously-growing list of ordered records (blocks). Once a block is added to the chain, it cannot be modified anymore. Furthermore, since blockchain is distributed, every node in the network has the same blockchain. According to Kelvin-san, these specifications make blockchain secure.


As usual, we had a party afterwards :)