Geo-Routing (Case Study on Craigslist) When you open craigslist.org it does a couple of redirection before finally landing on the subdomain closest to your city. Having separate sub-domain and listings for those particular cities is partly how it seems to handle the distribution of the content. (making it more scalable) Give it a try: https://craigslist.org Here is the journey of the request: curl -I craigslist.org HTTP/1.1 301 Found Location: https://www.craigslist.org/ redirected to https://www.

Trend of LLM Large Language Models continue to find new use cases. Though they started off as smart chatbots developers are using the open source models (thanks to corporates like Meta who open source models and no thanks to OpenAI) and fine tuning them to be used for specific domains. It’s so obvious that in the future these LLM are going to be our personal copilot, our wingman each tailored to each individual.

Literate Programming Python notebooks are so ubiquitous in the Machine learning community. All the cloud providers have dedicated services to run these notebooks: Jupyter on AWS, Jupyter on Azure, and Google Collab. These are popular largely because of how we can accommodate both code and instructional text in the same file. With each individual cell as an executable code block or a properly formatted markdown text, this makes it much easier to present information, document the information and make it a better educational material.

Query Planning Every SQL database (postgres,mysql) has a query planner built into it whose job is to find the best strategy to come up with the results given the query, its predicates and other expressions. Following examples are from postgres where I explore around how query planner makes its decision based on the data size in the table and the state of the database even in the presence of index.

Taskwarrior Taskwarrior is an OSS CLI (to make it fancier, let’s call it TUI - terminal UI) based task manager. I have been using it for sometime and it works great for my use case. My Earlier Post on setting GUI notifications I wanted some discipline in progressing with my side projects most of which are on github and here’s what I did: set up a script which overrides task warrior’s data directory to current directory if it’s a git repository, this is so that I can segregate my todo tasks

SQL Injection Starting off with one of my favorite xkcd comic SQL Injection is a way where a user is able to send some ‘code’ as part of the input data to the server all the way back to the database. (UI layer doesn’t sanitize/validate, API layer let’s the malicious ‘code’ reach the database because it’s not mindful in how to separate ‘query’ and ‘data’). Taking the example from xkcd:

Currying Currying is an idea of decomposing a function and making the expression more clearer. (Note: it’s just expression decomposition and not execution) Example In Go: package main import ( "fmt" "strings" ) func main() { fmt.Println(someComplexOperation("hello", "there", "programmer")) fmt.Println(someComplexOperation("hello", "there", "manager")) fmt.Println(someComplexOperation("hello", "there", "CEO")) // Currying this function // By looking at the fuction we see that it can be decomposed, first step of computing from first 2 arguments // can be done and reused.

Per Iteration Vs Per Loop In most of the languages, the loop variable (iterator) has a per iteration scope, meaning in each iteration a new copy of that variable is made. But in golang it’s the same variable that has the scope throughout the loop. All the changes (increments) apply to the same placeholder and if you are temporarily capturing them and spawning go routines or creating a function placeholder (for later execution) you start to see unexpected consequences.

Embeding a struct (just writing the type) creates a “has a relationship” with the parent. If the child implements any interface parent can directly invoke those methods. If there are more than one embedding and they implement a common method it’s okay as long as we don’t attempt to directly invoke it on parent (trying to do so results in compilation error - ambiguous selector) While encoding json (marshalling), encoder internally checks if the type implements json.

I have been using zsh (Z-shell) for a long time. Recently it came to my attention that opening a new terminal was taking about 2-3 seconds which is quite a long time. Just to rule out some other reasons, I switched my default shell to bash and saw if it was still slower and turned out that was not the case. Changing default shell chsh -s /bin/bash Normally all shells load up some configuration, aliases, etc while opening a session, .

Problem Recently I came across a scenario at work where I had to deal with real time updates of payment status in mobile which involved an asynchronous chain of events in the backend. Mobile would call the backend to initiate a payment request, backend would do basic validation - initialize the payment and return a token to the UI. Actual payment involved multiple API calls to some of the internal systems as well as third party systems.

DB Connections are expensive Normally following things happen when your backend service talks to a database: service initiates a connection to DB using the database driver db does the authentication and establishes a network session (tcp) if succeeded program/service performs some db operation (CRUD) which follows authorization (if user/role is allowed to perform the requested operation) Now if the service decides to close the connection, it has to do these all over again which is quite an expensive process.

We have pretty powerful modern IDE and tooling around debugging (including structured logging and so on). It’s so easy to yield to this paradigm of programming. Now I think it’s quite a bad thing. Here are my reasons: (disclaimer - my reasons against debugging only when considered as a primary approach to coming up with solution) too much reliance on debugging (if something goes wrong step through) takes away a key thing (thinking from first principles).

I have been working as a professional software engineer for over 4 years now and along the way I have picked up some heuristics about how should I take the task in hand. Clarification Getting a crystall clear clarity of what you are going to implement is of paramount importance. Pay as much attention to details as possible. It’s okay to conduct few more meetings with teammates on getting clarification rather than doing a root cause analysis for a misunderstood or a buggy code down the line.

AWS Cloudwatch cloudwatch allows to take a peek into your aws applications, their logs and other metrices. Apart from the centralized logging (kibana,loggly) where we limit retention duration (cost factor), cloudwatch provides a way to do pay-per-volume query operations. AWS CLI installation on mac configuration aws configure Setting up AWS keys Although cloudwatch comes with a nice GUI, we can leverage CLI to automate and ease up some tasks. Here is a use case for filtering logs and downloading them for further analysis.

up arrow key takes you to a list of recently used commands history command lists out the history of your commands (size as configured) you can search in your history by using (ctr-R), helpful if you know some keyword You can tag your commands with comments for ease with history search eg : some esoteric command #fixes-all (You can reverse search using fixes-all) all your commands in history are numbered

Concurrecy Vs Parallelism (Go-lang Example) This is the general understanding of concurrency vs parallelism. Parallelism: Having multiple threads do similar task which are independent of each other in terms of data and resource that they require to do so. Eg: Google crawler can spawn thousands of threads and each thread can do it’s task independently. Concurrency: Concurrency comes into picture when you have shared data, shared resource among the threads. In a transactional system this means you have to synchronize the critical section of the code using some techniques like Locks, semaphores, etc.

Closure The inner function has access to variables which were in the enclosing scope even after that ’ennclosing scope’ execution is over. (Inner function gets a copy) This is my attempt to understand why is this such an important concept in functional programming, why is it named so and so on… Functions are first class citizens In functional paradigm of programming functions are considered as ‘first class citizens’. This means functions can be

Microservice is not the silver bullet Software Industry is heavily trend-driven. People love going behind the trendy hypes microservices are cool containers are cool noSQL is cool this new language/framework is cool what makes a good use case for microservice? functionality has very clear demarcation/boundary (the most important.. don’t try to force split an already domain complex module) functionality needs independent scaling functionality needs lot of re-use (many components need it) functionality demands language agnosticity (rapid iterations, deployment) what should you ensure?

What are the things that should be on your checklist when you are facing a situation where your db queries are taking a lot of time? Cross check simple things such as if you would require pagination. If your filters are none (or too broad) even with optimized query plan/indexing query can take some time. Take the slow query, view the execution plan and cross check if you would require indexing, if your query is set to run with the indices (eg: is the query properly making use of functional indices?

Why Some ISPs block TCP 22? Recently I got to know that some ISP indeed block any outbound traffic for port 22 (tcp) which is a standard port for ssh connections. (I came to know about this as I was being unable to log into one of the compute instances from my terminal via gcloud in gcp. I prefer terminal logins instead of browser as it’s simply 100 times easier to work with.

Arrays Arrays are contiguous locations of memory set aside during compilation. (memory allocated is virtual address which is later mapped to actual memory address during run-time) In the most simple form you need a base address and you can read starting from there. (there’s a limit how far you can go even with languages like C) Simple Integer Array in C: int arr[] = {1,2}; for (int i=0;i<10;i++){ printf("%d\n",arr[i]); } C doesn’t have a way to have a check on how far to go.

This was primarily for the inconvenience I had faced a few days prior java-storage API returns null for object with space in name Fixing this issue required a bump in version Upgraded version required an additional dependency which wasn’t documented. I added it. (This is a rather lame contribution but I hope it gives me an impetus for trying and exploring more)

CMD Vs Entrypoint is a bit confusing at the beginning. When should you use CMD and when should you use ENTRYPOINT while building your docker images? CMD This is the command that runs when you start a container. Eg: cmd-docker FROM ubuntu CMD [ "echo" , "this is running from CMD which runs when the container starts" ] Building an image: docker build -t cmd-docker -f cmdvsentrypoint.yaml . Default docker run:

Tuesdays with Morrie I loved the narration. I loved the content and the insights this book provides in addition to the beautifully crafted chapters. Almost every page gets you into a contemplative mode of introspection about life, love and the ‘culture’. The biggest take away for me from this book is this: Culture is a man-made thing and is presented in such a way that people tend to conform to it.

Accessing gcp compute instances via gcloud sdk You have a bunch of options to ‘ssh’ into gcp VM instances: open in browser window open in browser window on custom port open in browser window using provided ssh key another ssh client (putty, termius, etc) I personally find the ‘gcloud sdk’ way much more convenient to use since I can just launch it from my terminal without having to open the browser, log into cloud console, locate the vm instance and then ssh’ing into it.

Problem Solving is the gold We all focus on problem solving. We are in pursuits of so many activities which aim to identify the problems, propose a solution to them, build the solution and hopefully create some value out of it. But what is good problem solving? Especially in the realm of computer science. The greatest part in ‘problem solving’ is not actually the solution. Its the break down of the problem.

Sample github action for gradle build

Migration to Cloud As your applications scale it becoomes tedious, expensive and error prone to have it all on your data centers. Modern microservices are built with the philosophy of isolation. Scaling up and down of those services should happen with a snap of a finger (enter containerization technologies like dockers replacing traditional VMs), you scale up to serve bigger traffic and also to make it fault tolerant. But even if you have 20 docker pods, a failure in data center as simple as switch failure can bring your whole infrastructure to its knees.

TaskWarrior CLI TaskWarrior is a CLI based task manager which I find pretty handy to use. When you are working on something (where your linux terminal is always with you), you can just create an interactive todo from your command line. ~/sudip/sudipbhandari126.github.io(master*) » task add write a blog about task warrior reminder sudipbhandari@sudipbhandari-Latitude-5480 Created task 1. ---------------------------------------------------------------------------------------------------------------------------------------------- ~/sudip/sudipbhandari126.github.io(master*) » task list sudipbhandari@sudipbhandari-Latitude-5480 ID Age Description Urg 1 3s write a blog about task warrior reminder 0 1 task ---------------------------------------------------------------------------------------------------------------------------------------------- ~/sudip/sudipbhandari126.