Blogs

RAG (Retrieval-Augmented Generation) Introduction RAG is a system that can provide new knowledge to the model. flowchart LR A1[1. Data Extraction] subgraph RAG_Pipeline B2[2. Chunking] B3[3. Vectorization / Embedding] B4[4. Index Creation] B5[5. Retrieval] end C7[7. Reranking] D9[9. LLM Summarization / Generation] E6[6. Retrieval Evaluation] F8[8. Rerank Evaluation] G10[10. Final Evaluation] A1 --> B2 B2 --> B3 B3 --> B4 B4 --> B5 B5 --> C7 C7 --> D9 B5 -. evaluate .-> E6 C7 -. evaluate .-> F8 D9 -. evaluate .-> G10 E6 --> RAG_Pipeline F8 --> RAG_Pipeline G10 --> RAG_Pipeline There are three main mode for AI systems: ...

System design for LLM and RAG project

Large-scale machine learning design

In this post, we will explore the system design of a financial data ingestion system. This system is crucial for collecting, processing, and storing financial data from various sources. There can be many conplicated distributed systems to handle financial data ingestion, but we will focus on a simplified version that can be used as a starting point for building such systems. System Design Overview For financial data ingestion, we need to consider generally two typpes of data: Metadata and Time Series Data. ...

Some famous algorithms for interview preparation.

Basic concepts of distributed system.

NLP notes

Many people are interested in investing, but before doing investments, it is better to understand some basic concepts of the financial world. Investment is the act of allocating resources, usually money, in order to generate income or profit. It can take many forms, such as stocks, bonds, real estate, or mutual funds. In another word, investment is the use of assets to earn income or profit. Investment forms Stocks, bonds, real estate, and mutual funds are some of the most common forms of investment. Each has its own characteristics, risks, and potential returns. ...

Web scraping with Selenium on remote server.

Python related notes.