Below you will find pages that utilize the taxonomy term “engineering”
Posts
Geo Routing (Craigslist Example)
Geo-Routing (Case Study on Craigslist) When you open craigslist.org it does a couple of redirection before finally landing on the subdomain closest to your city. Having separate sub-domain and listings for those particular cities is partly how it seems to handle the distribution of the content. (making it more scalable)
Give it a try: https://craigslist.org
Here is the journey of the request:
curl -I craigslist.org HTTP/1.1 301 Found Location: https://www.craigslist.org/ redirected to https://www.
read morePosts
Quirks of Query Planner
Query Planning Every SQL database (postgres,mysql) has a query planner built into it whose job is to find the best strategy to come up with the results given the query, its predicates and other expressions. Following examples are from postgres where I explore around how query planner makes its decision based on the data size in the table and the state of the database even in the presence of index.
read morePosts
Using taskwarrior to stay disciplined with side projects
Taskwarrior Taskwarrior is an OSS CLI (to make it fancier, let’s call it TUI - terminal UI) based task manager. I have been using it for sometime and it works great for my use case.
My Earlier Post on setting GUI notifications
I wanted some discipline in progressing with my side projects most of which are on github and here’s what I did:
set up a script which overrides task warrior’s data directory to current directory if it’s a git repository, this is so that I can segregate my todo tasks
read morePosts
A case of slow loading zsh
I have been using zsh (Z-shell) for a long time. Recently it came to my attention that opening a new terminal was taking about 2-3 seconds which is quite a long time.
Just to rule out some other reasons, I switched my default shell to bash and saw if it was still slower and turned out that was not the case.
Changing default shell chsh -s /bin/bash Normally all shells load up some configuration, aliases, etc while opening a session, .
read morePosts
Websockets, Dealing with realtime updates for async API
Problem Recently I came across a scenario at work where I had to deal with real time updates of payment status in mobile which involved an asynchronous chain of events in the backend. Mobile would call the backend to initiate a payment request, backend would do basic validation - initialize the payment and return a token to the UI. Actual payment involved multiple API calls to some of the internal systems as well as third party systems.
read morePosts
Database Connection Pooling
DB Connections are expensive Normally following things happen when your backend service talks to a database:
service initiates a connection to DB using the database driver
db does the authentication and establishes a network session (tcp) if succeeded
program/service performs some db operation (CRUD) which follows authorization (if user/role is allowed to perform the requested operation)
Now if the service decides to close the connection, it has to do these all over again which is quite an expensive process.
read morePosts
Debugging driven development (is bad)
We have pretty powerful modern IDE and tooling around debugging (including structured logging and so on). It’s so easy to yield to this paradigm of programming. Now I think it’s quite a bad thing. Here are my reasons:
(disclaimer - my reasons against debugging only when considered as a primary approach to coming up with solution)
too much reliance on debugging (if something goes wrong step through) takes away a key thing (thinking from first principles).
read morePosts
How I work
I have been working as a professional software engineer for over 4 years now and along the way I have picked up some heuristics about how should I take the task in hand.
Clarification
Getting a crystall clear clarity of what you are going to implement is of paramount importance. Pay as much attention to details as possible. It’s okay to conduct few more meetings with teammates on getting clarification rather than doing a root cause analysis for a misunderstood or a buggy code down the line.
read morePosts
Accessing Cloudwatch logs via AWS CLI
AWS Cloudwatch cloudwatch allows to take a peek into your aws applications, their logs and other metrices. Apart from the centralized logging (kibana,loggly) where we limit retention duration (cost factor), cloudwatch provides a way to do pay-per-volume query operations.
AWS CLI installation on mac
configuration aws configure Setting up AWS keys
Although cloudwatch comes with a nice GUI, we can leverage CLI to automate and ease up some tasks. Here is a use case for filtering logs and downloading them for further analysis.
read morePosts
Some notes on HTTP
Learning from the udacity course on client server communication
course link: https://classroom.udacity.com/courses/ud897
Using netcat:
nc google.com 80 <head/get/post/put> headers: OPTIONS: get a list of methods supported by the URL end point HEAD: just the header and not the body.. validate if the incoming data will fit the memory, cache validation xml-http-request: xhr, traditional way of doing ajax, new way is javascript promises which are chainable callbacks and are asynchronous
How head is used for cache checking.
read morePosts
Database Connection Pooling
DB Connections are expensive Normally following things happen when your backend service talks to a database:
service initiates a connection to DB using the database driver db does the authentication and establishes a network session (tcp) if succeeded program/service performs some db operation (CRUD) which follows authorization (if user/role is allowed to perform the requested operation) Now if the service decides to close the connection, it has to do these all over again which is quite an expensive process.
read more