We must cultivate our garden. Volatire (Candide) This is the closing sentence in Volatire’s Satire Candide. Of course the metaphorical sense is, regardless of what’s happening in the world we should continue to work on ourselves and shouldn’t worry too much in things which are either inconsequential or are out of our controls. This doesn’t mean we become ignorant or indifferent towards those but rather put oneself and one’s concerns first.

Is Go Fast? Go is faster than interpreted languages like python in terms of sheer execution since it’s compiled to native executable format. Go is slower than other compiled languages like C,Rust in terms of execution. This is primarily because although compiled go runs with a ‘runtime’ which does some housekeeping (scheduling and multiplexing goroutines, managing/freeing memory (GC) etc). This overhead is much smaller than a virtual machine required for languages like java but is an overhead nonetheless.

Logs Logs (in a very degenerative sense) are a series of events/statements that act like the breadcrumb for the things that happened. Degenerative as I am referring mostly as application or software logs. (More pure meaning would be let’s say database logs which are used for replication process or a series of events for active state machine replication). I want to talk about the logs (application logs, system logs). For interested reader, I highly recommend I heart logs which talks about the power of logs in a distributed systems - how a series of immutable logs help us chronologically capture a series of events and computing/coming to consensus/committing transaction etc on top of it.

Go’s approach to Object Go is a fairly simplistic language and doesn’t come with inbuilt classes and other features that most object oriented programming languages have. It gives you building blocks in the form of structs and it’s upto programmer to design the abstraction around it. There is no inheritence - Go takes a very strong stance on composition over inheritence camp. (even other languages have practitioners who preach composition over inheritence but it’s not strictly enforced).

I encountered this quote while reading Martin Klepmann’s Designing Data-Intensive Application which got me thinking: A complex system that works is invariably found to have evolved from a simple system that works. The inverse proposition also appears to be true: A complex system designed from scratch > never works and cannot be made to work. John Gall, Systemantics (1975) This is a famous “law” in software engineering. (although the original context in which the author said this is in regards to general systems (mechanical systems like Titanic ship or human systems like an organization; come to think of it, even software systems are just an extension of such systems).

Time is not just an abstraction Of course time as we know in a day to day sense is an abstract thing. It’s a concept we defined to measure something. (like we defined length to measure how long something is or weight to define how heavy something is). But behind that abstraction is a physical reality. Events are a manifestation of that reality which really give us a sense of time.

Geo-Routing (Case Study on Craigslist) When you open craigslist.org it does a couple of redirection before finally landing on the subdomain closest to your city. Having separate sub-domain and listings for those particular cities is partly how it seems to handle the distribution of the content. (making it more scalable) Give it a try: https://craigslist.org Here is the journey of the request: curl -I craigslist.org HTTP/1.1 301 Found Location: https://www.craigslist.org/ redirected to https://www.

Trend of LLM Large Language Models continue to find new use cases. Though they started off as smart chatbots developers are using the open source models (thanks to corporates like Meta who open source models and no thanks to OpenAI) and fine tuning them to be used for specific domains. It’s so obvious that in the future these LLM are going to be our personal copilot, our wingman each tailored to each individual.

Literate Programming Python notebooks are so ubiquitous in the Machine learning community. All the cloud providers have dedicated services to run these notebooks: Jupyter on AWS, Jupyter on Azure, and Google Collab. These are popular largely because of how we can accommodate both code and instructional text in the same file. With each individual cell as an executable code block or a properly formatted markdown text, this makes it much easier to present information, document the information and make it a better educational material.

Query Planning Every SQL database (postgres,mysql) has a query planner built into it whose job is to find the best strategy to come up with the results given the query, its predicates and other expressions. Following examples are from postgres where I explore around how query planner makes its decision based on the data size in the table and the state of the database even in the presence of index.

Taskwarrior Taskwarrior is an OSS CLI (to make it fancier, let’s call it TUI - terminal UI) based task manager. I have been using it for sometime and it works great for my use case. My Earlier Post on setting GUI notifications I wanted some discipline in progressing with my side projects most of which are on github and here’s what I did: set up a script which overrides task warrior’s data directory to current directory if it’s a git repository, this is so that I can segregate my todo tasks

SQL Injection Starting off with one of my favorite xkcd comic SQL Injection is a way where a user is able to send some ‘code’ as part of the input data to the server all the way back to the database. (UI layer doesn’t sanitize/validate, API layer let’s the malicious ‘code’ reach the database because it’s not mindful in how to separate ‘query’ and ‘data’). Taking the example from xkcd:

Currying Currying is an idea of decomposing a function and making the expression more clearer. (Note: it’s just expression decomposition and not execution) Example In Go: package main import ( "fmt" "strings" ) func main() { fmt.Println(someComplexOperation("hello", "there", "programmer")) fmt.Println(someComplexOperation("hello", "there", "manager")) fmt.Println(someComplexOperation("hello", "there", "CEO")) // Currying this function // By looking at the fuction we see that it can be decomposed, first step of computing from first 2 arguments // can be done and reused.

Per Iteration Vs Per Loop In most of the languages, the loop variable (iterator) has a per iteration scope, meaning in each iteration a new copy of that variable is made. But in golang it’s the same variable that has the scope throughout the loop. All the changes (increments) apply to the same placeholder and if you are temporarily capturing them and spawning go routines or creating a function placeholder (for later execution) you start to see unexpected consequences.

Embeding a struct (just writing the type) creates a “has a relationship” with the parent. If the child implements any interface parent can directly invoke those methods. If there are more than one embedding and they implement a common method it’s okay as long as we don’t attempt to directly invoke it on parent (trying to do so results in compilation error - ambiguous selector) While encoding json (marshalling), encoder internally checks if the type implements json.

I have been using zsh (Z-shell) for a long time. Recently it came to my attention that opening a new terminal was taking about 2-3 seconds which is quite a long time. Just to rule out some other reasons, I switched my default shell to bash and saw if it was still slower and turned out that was not the case. Changing default shell chsh -s /bin/bash Normally all shells load up some configuration, aliases, etc while opening a session, .

Problem Recently I came across a scenario at work where I had to deal with real time updates of payment status in mobile which involved an asynchronous chain of events in the backend. Mobile would call the backend to initiate a payment request, backend would do basic validation - initialize the payment and return a token to the UI. Actual payment involved multiple API calls to some of the internal systems as well as third party systems.

DB Connections are expensive Normally following things happen when your backend service talks to a database: service initiates a connection to DB using the database driver db does the authentication and establishes a network session (tcp) if succeeded program/service performs some db operation (CRUD) which follows authorization (if user/role is allowed to perform the requested operation) Now if the service decides to close the connection, it has to do these all over again which is quite an expensive process.

We have pretty powerful modern IDE and tooling around debugging (including structured logging and so on). It’s so easy to yield to this paradigm of programming. Now I think it’s quite a bad thing. Here are my reasons: (disclaimer - my reasons against debugging only when considered as a primary approach to coming up with solution) too much reliance on debugging (if something goes wrong step through) takes away a key thing (thinking from first principles).

I have been working as a professional software engineer for over 4 years now and along the way I have picked up some heuristics about how should I take the task in hand. Clarification Getting a crystall clear clarity of what you are going to implement is of paramount importance. Pay as much attention to details as possible. It’s okay to conduct few more meetings with teammates on getting clarification rather than doing a root cause analysis for a misunderstood or a buggy code down the line.

AWS Cloudwatch cloudwatch allows to take a peek into your aws applications, their logs and other metrices. Apart from the centralized logging (kibana,loggly) where we limit retention duration (cost factor), cloudwatch provides a way to do pay-per-volume query operations. AWS CLI installation on mac configuration aws configure Setting up AWS keys Although cloudwatch comes with a nice GUI, we can leverage CLI to automate and ease up some tasks. Here is a use case for filtering logs and downloading them for further analysis.

up arrow key takes you to a list of recently used commands history command lists out the history of your commands (size as configured) you can search in your history by using (ctr-R), helpful if you know some keyword You can tag your commands with comments for ease with history search eg : some esoteric command #fixes-all (You can reverse search using fixes-all) all your commands in history are numbered

Concurrecy Vs Parallelism (Go-lang Example) This is the general understanding of concurrency vs parallelism. Parallelism: Having multiple threads do similar task which are independent of each other in terms of data and resource that they require to do so. Eg: Google crawler can spawn thousands of threads and each thread can do it’s task independently. Concurrency: Concurrency comes into picture when you have shared data, shared resource among the threads. In a transactional system this means you have to synchronize the critical section of the code using some techniques like Locks, semaphores, etc.

Closure The inner function has access to variables which were in the enclosing scope even after that ’ennclosing scope’ execution is over. (Inner function gets a copy) This is my attempt to understand why is this such an important concept in functional programming, why is it named so and so on… Functions are first class citizens In functional paradigm of programming functions are considered as ‘first class citizens’. This means functions can be

Microservice is not the silver bullet Software Industry is heavily trend-driven. People love going behind the trendy hypes microservices are cool containers are cool noSQL is cool this new language/framework is cool what makes a good use case for microservice? functionality has very clear demarcation/boundary (the most important.. don’t try to force split an already domain complex module) functionality needs independent scaling functionality needs lot of re-use (many components need it) functionality demands language agnosticity (rapid iterations, deployment) what should you ensure?

What are the things that should be on your checklist when you are facing a situation where your db queries are taking a lot of time? Cross check simple things such as if you would require pagination. If your filters are none (or too broad) even with optimized query plan/indexing query can take some time. Take the slow query, view the execution plan and cross check if you would require indexing, if your query is set to run with the indices (eg: is the query properly making use of functional indices?

Why Some ISPs block TCP 22? Recently I got to know that some ISP indeed block any outbound traffic for port 22 (tcp) which is a standard port for ssh connections. (I came to know about this as I was being unable to log into one of the compute instances from my terminal via gcloud in gcp. I prefer terminal logins instead of browser as it’s simply 100 times easier to work with.

Arrays Arrays are contiguous locations of memory set aside during compilation. (memory allocated is virtual address which is later mapped to actual memory address during run-time) In the most simple form you need a base address and you can read starting from there. (there’s a limit how far you can go even with languages like C) Simple Integer Array in C: int arr[] = {1,2}; for (int i=0;i<10;i++){ printf("%d\n",arr[i]); } C doesn’t have a way to have a check on how far to go.

This was primarily for the inconvenience I had faced a few days prior java-storage API returns null for object with space in name Fixing this issue required a bump in version Upgraded version required an additional dependency which wasn’t documented. I added it. (This is a rather lame contribution but I hope it gives me an impetus for trying and exploring more)

CMD Vs Entrypoint is a bit confusing at the beginning. When should you use CMD and when should you use ENTRYPOINT while building your docker images? CMD This is the command that runs when you start a container. Eg: cmd-docker FROM ubuntu CMD [ "echo" , "this is running from CMD which runs when the container starts" ] Building an image: docker build -t cmd-docker -f cmdvsentrypoint.yaml . Default docker run: