Thinking About Microservices Architecture

In the past 2 weeks I was so busy with many things. Had an interview with a well known media company. It was the longest interview process that I ever had.  and I was busy finishing a system I’m developing and collecting data for the green system here.

Right now, at AiU, I have to think a lot about how to implement IoT solution. How the overall of the architecture looks like.

And I chose microservice architecture.

From microservices.io:

Microservices – also known as the microservice architecture – is an architectural style that structures an application as a collection of loosely coupled services, which implement business capabilities. The microservice architecture enables the continuous delivery/deployment of large, complex applications. It also enables an organization to evolve its technology stack.

Why do I choose microservices architecture?

At AIU, I won’t be connecting just one type of device (the switch), but I’ll be connecting with the door and the solar, the battery and the grid.

I don’t like a single point of failure even though using just one system may make the system easier to make.

Another challenge that I have is that all these devices “speak” different languages. One is Java, another is C++ and another… Microsoft Access.

Since I have to connect to multiple types of devices with multiple languages at the same time, I doubt that a monolithic architecture is a good way to integrate all these devices.

From Wikipedia, monolithic applications are:

In software engineering, a monolithic application describes a single-tiered software application in which the user interface and data access code are combined into a single program from a single platform.

Drawbacks of Microservices Architecture

  1. To debug is quite hard. You have to write all the loggings for all the different services and it may have different languages
  2. Consumes more time to develop than a monolithic one. Each type of devices has its own so-called “controllers”. Instead of one big one that manages everything.

Pros of Microservices Architecture

  1. Easier to add new functions. If I wanted to add new function to the smart switch like scheduling, I just whip open the smart switch controller’s project, I can add that function.
  2. Know exactly what’s going on – instead of guessing, I know which device caused problem. Can’t get the data from solar panel, OK it’s autoswitch controller’s problem. I can debug from there.

Microservices is not a silver bullet to me. It’s just that I felt that using this architecture is the best for this situation of mine.

How Hacker News Will Affect Your Site?

I have never expected that this little blog of mine will reach the 1st page of Hacker News.

Yeay! In Hacker News!

I dreamed of it. How wonderful it would be if I just reached the 1st page of Hacker News… but I never think that I’ll get to the 1st page.

Just out of curiosity, I submitted the post I wrote few months back to Hacker News. It’s basically a place where startup people read whatever happening in the startup world.

And a few minutes after I went away from my laptop to watch TV, I refreshed my blog.

And it’s Error 500.

And MySQL is down.

Ops. Oh oh.

Then I realised: my site IS trending up in Hacker News.

I’m elated. Electrified is the right word I guess?

So many people want to see what the fuss is all about the blog post but it’s not up?

So how do I get my site back up running again?

And… I did a classic mistake that many made.

I did not activate caching plugin.

Nope. I did not even download it.

What a foolish mistake to make.

I thought of enabling it, but I never bothered to download it in the first place.

I opened the Hacker News link and saw so many people bashing me. (Right now most of the comments have been deleted).

Damn.

I tried rebooting my droplet (I use DigitalOcean)… but it didn’t help much. There is just so many people coming in, even when I restarted it, the MySQL service instantly crashed.

ERROR 2002 (HY000): Can’t connect to local MySQL server through socket ‘/var/run/mysqld/mysqld.sock’ (2)

No matter how many times I restarted it, the service keeps on failing.

Since I’m unable to get the site up running as usual, I do what I can do: put a placeholder page until I get everything up and running again.

So, I opened up Google cache (thanks Google!) and copied and pasted the article in a HTML file that I did in a haste. Bad formatting and all. Ah, just put it up so that at least people can see it when they opened the blog.

It’s not pretty but at least people can see it, right?

Instead of being greeted by server 500 error, I greet visitors who is coming to my blog with this ugly looking HTML page.

That’s a quite good lesson to learn. I need to have a static site as a failover just in case your WordPress site is down.

I mean, who knows, right? Especially with a small blog that is running on 512MB VM.

Oh well, that’s a start. At least it will free the MySQL from having to serve the people, right?

I waited a bit.

I restarted the droplet again and tried to enter the admin panel again.

Yeay! I can enter the admin panel!

But usually it will crash in just a few seconds after I entered it.

It’s not successful but at least it’s a start.

I replaced the front page with the same HTML and managed to get to the Plugins page.

I added WP Fastest Cache and enabled the following settings:

Not sure if settings I set is the best though. If you have a suggestion, do leave a comment!

After that I asked around for advice. I spammed Malaysian Open Source (Sumber Terbuka Malaysia) Telegram group and my friends for any advice they can give for someone in my situation.

One of the tips given by members of that group is I can just use CDN provided by Cloudflare for free. Why, in God green earth did I forgot about that?

That’s the next thing that I did.

I signed up for Cloudflare, followed the instruction, changed the nameservers to Cloudflare’s and also searched for Cloudflare plugin for WP if it exists. (It did, and it is developed by Cloudflare team).

By now, this blog was routinely restarted again and again.

I remembered using MySQL tuner to optimize an exam system that I helped configure and run for my current workplace (Albukhary International University).

So I used it on my blog.

But the sad part is I am running on the lowest plan so most of the suggestions are plain useless. I can’t use it.

I asked myself, can I just upgrade the droplet for a while? The RAM and the CPU just for a little while when the going gets tough?

It turns out that I can do it in DigitalOcean!

So yeah, what I do was, I powered off the droplet and then, I increase the CPU and RAM only. I use the $20/month option.

It’s getting smoother now.

I boot up the droplet and immediately feel the responsiveness of the blog. Right now I am able to browse the admin panel without having the dreaded server 500 error.

So right now, it’s a combination of caching, CDN and just a raw horsepower from bigger CPU and RAM.

How was the traffic, really?

The traffic surge from being in the Hacker News will last for 1 or 2 days.

As you can see from the graph above, I did not get much traffic after 1 or 2 days.

But you may get new links from unexpected places. Other blogs linking to you or people shared it through their social media account.

So you may get residual traffic from other websites too.

For example, I do get traffic from Reddit.

That’s good for SEO I guess. Google always love links. Lots of it, especially.

For someone who is accustomed with not being seen by anyone, scaling is hard. I have never expected my site to serve thousands of users in under few minutes.

After the fact

After all the dust has settled, I read all the comments at Hacker News that I linked in the first paragraph. A lots of comments… and there are some good advices too!

One of the advice given is to use load testing tool. One of the tools linked is Loader.io. I have never encountered this before, so maybe the next time I ever send my article to Hacker News, I’ll try to use this service first and see how it goes.

And I want to say thanks to all the tips given by members of Open Source Malaysia Telegram group. Another thing that they did was they also told me if my site was down from their sides, which is good. At least I have a realtime feedback of what’s going on from another visitor point of view.

I also want to thank a friend who proofread my blog post few hours after the blog post was hit by Hacker News effect. Or else, the article would read even worse.

Some of the plugins that I mentioned above but too lazy to link to can be easily searched from WordPress Add New Plugin page.

What I Learned Today While Making IOS Apps?

So today I learned how to use Page View Controller. It’s kinda different than making app for Android. You know, to make a pager kind of style.

Right now I am googling more and more about it. Right now, more often than not, I am trying to find if there is any similarities between using Java and Swift.

There are obviously some kind of similarities but there are a lot of differences.

Both are object oriented. (If my memory serves me correctly), but the important thing is: there’s no semicolon to end a statement! Hurray!

I also learned to play around with Cocoa Pods. How to install it. Just managed to get Google Maps SDK for IOS. Let’s play with Google Maps after this.

Today is Swift Day

At least for me. Today I am playing around with making iOS app. Making an app for IOS is a bit different than making for Android.

For starter, to develop a native app in Android, you would be using Java or Kotlin.

To make apps for IOS, you have to use either Objective-C or Swift.

The layout is also different. There is “Auto Layout” feature in XCode, but in Android you have more flexibility on how you want to achieve your desired layout.

Sometimes constraint is good too: you have to be creative with what you have. Hence you see the simplicity of IOS apps but it felt premium.

How to Create Web Service In Less Amount Of Time?

I want to talk to you about the stupid thing I do. How to avoid it.

I always create a HTML form to test all kinds of JSON response for the web service.

It’s easy, but it’s redundant.

I don’t need to do that every time.

Actually I can just use this: Advanced REST Client.

It is a Chrome app which you can download it into your browser.

So now, you can test your JSON response or you can test if you properly set your variable name and got the response you want.

You don’t need to recreate a form every time you want to test whether you named your variables properly or not.

Download it, use it. Maybe in the future I will create a tutorial based on that.

Malaysian Programmers Blog List

So how many programmers in Malaysia have their own blog?

In this post, I am going to put up a list of blogs that are written by Malaysian. It can be in either English or Malay.

Crynobone – by Mior Muhammad Zaki

Ful-Lamak! – by Saiful Yusoff

Shahrul Nizam – by Shahrul Nizam

Nasrul Hazim Mohamad – by Nasrul Hazim Mohamad

Blog Pisyek – by Pisyek Kumar

Amirul Abu – by Amirul Abu

Canang Technologies – by Faizal Abdul Manan

Malaysian Developer – by Nurul Haszeli Ahmad

Kerul.net – by Khirulnizam Abd Rahman

Software Development the Hard Way – by Hakim Ahmad

Mazamyblog – by Mohammad Aiman

The Kotak – by Umar Luqman

Ijat.my – by Ijat

Response.Write() – by Sharuzzaman Ahmat Raslan

me, myself & mbek – by Hairul Mbek Moktar

Integra Solid – by Muhammad Fathur Rahman

Rizal Mohd Nor – by Rizal Mohd Nor

fajarhac – by Fakhrullah Padzil

What I’ve Learned So Far As a Programmer?

Previously I wrote about how I learned programming. My history with programming.

So today, I would like to share with you what I learned so far as a programmer. Actually most of the lessons I learned in the past 2 years made me realize that I learned more in the past 2 years than many of the years before.

  • Try to learn a new programming language every year

… or at least “dive deep” into learning all features of the language you already know right now

This is important. One of the things I learned this 2 years is that you have to learn something new every year in order to keep up to date. It doesn’t necessarily mastering a new language, but at least try it so that you will be able to expand your understanding of how to program things.

  • Library can come and can go in less than one year…

But language usually lives much longer.

Usually this means that the library is not maintained anymore. This happens when there’s only one maintainer. This usually happened in Android where there are rapid changes from one version to another.

Language on the other hand usually have multiple contributors or maintainers. So they usually lived longer.

  • Learn to learn quickly

This got to be the most important thing I have ever learned. The faster I learn, the more I will be able to create.

Learning quickly is probably one of the single biggest multiplier effect that I can have in finishing project I’m involved in.

  • Try to know exactly what you don’t know…

So that you’ll be able to quickly find a solution for that.

When you are able to learn quickly, you will be able to know what you don’t know. And it become sort of a loop. You’ll be able to know what you want to look for in Google.

  • It doesn’t matter if you don’t know algorithms…

But it doesn’t hurt to learn about it (because it might help you in the future).

This is important. Usually you don’t use algorithm in programming a system but if it’s something that will scale fast, your understanding of algorithm might be useful.

  • Identify exactly what you’re weak at…

And see if you could try to fix your weakness in your understanding of software engineering.

In self-help world you will hear people talk about focus on your strength but in order to develop a technical skill, you need to identify exactly what you are weak at.

For me, I am weak at algorithm because I don’t have a formal engineering background. I come from an information field where I learned mostly about records and library.

  • Break project down into manageable bits

To me, basically a project = big problem statement.

Sometimes you don’t need to do any programming if you have another way to solve that problem.

So you have to break it down into something that you understand. For instance if you want to make a simple archiving system, what do you need?

Obviously you would need a database. a way to create a record, to modify and delete it which is the standard CRUD thing to do.

So by breaking things down into smaller and smaller piece, you will be able to to code it and make the functions required to solve that big problem statement.

I guess that’s all for now. What I’ve learned so far in programming. Obviously I will have more things to learn in the future.

Until next time!

THE BIGGEST LESSON I LEARNED TODAY

Don’t forget to turn on your caching plugin and CDN. I submitted this blog post to Hacker News and this website can’t be accessed.

And I HOPE it won’t crash right after I wrote this. I hope! Haha.

Focus

I guess the image above shows what focus is all about.

If you are focused, you can go far. Your energy basically will be used in one direction. So you can go far. If not, you might go in so many direction and you will lose focus.

Those energy that you have will be diluted and in the end, you won’t be able to reap the benefit quickly enough just because you go into too many directions.

Consider Elon Musk.

He do it one by one. From one company to another company. From a weak standing to a strong one. He gathered small wins and In the end, he is able to make big wins. And big bets too.

But he only did it after he has all of those small (well, it is big, actually) wins.

X.com, PayPal, and all those success keeps piling up enabling him to eventually diversify himself into many businesses.