CLOUD

In the context of modern IR Systems, using a dictionary or hash table to represent inversion list would be kind of tedious as it request huge amount of memory. There for its obvious that we need to get the help of secondary storage to store the content and retrieve when required. Then again there is a huge overhead of using secondary storage to store and read a dictionary. So the solution that we came across was B-Tree, which is a used almost each and every implementation of inverted index. Definition of a B-Tree of order ‘m’ A root node with 2 ~ 2m keys All the other internal nodes have between m ~ 2m keys All keys are kept in ascending order All levels have the same level of differ (at most 1) e.g: Inversion lists structures are used because they provide optimum performance in searching large databases. The optimality comes from the minimization of data flow in resolving a query. Only data directly related to the query are retrieved from secondary storage. The beauty of B-Trees lie...

‘Google’ – One of the innovative and revolutionary concept which was blossomed in early ‘96, as a research project of Larry Page and Sergey Brin (Google’s President incharge of Technology). Google only consume a spam of 13 years to climb to the peek of internet search technology and now, its in a state which is almost untouchable and unreachable to the competitors. I recently watch a nice interview with Sergey Brin and its really awesome as it reveals some of interesting facts about Google, it’s challenges and Sergey him self. Google manufactures Nothing! Yeah… it manufactures nothing but produce an IDEA. The idea or the mission is, ‘Accumulate all the world’s information and make it accessible and useful to everyone’. This sounds like an non-commercial or non-profitable but we should keep in mind that there is a underlying business process which may based on several revenue sources. In general, “Search Engines” use one or more of the following revenue sources. Charge advertisers for p...

‘Secondary Storage’ is still a nightmare in achieving higher performance in modern computer systems and often the power of multi-core CPU is more or less negate due secondary storage low performance IO. Of course the performance of secondary storage IO has improved in the recent past but it is still inferior relative to CPU performances. A computer must retrieve an item and place it in main memory before it can be processed. In order to overcome the low performances of the system one must organize the files intelligently and making the retrieval efficient. The file organization depends on the retrieval method; sequential or random. Particularly secondary storage IO is a huge overhead in the context of random access method. Therefore associated with a large, randomly accessed file in a computer system is an index. An index is often a file that stored in the disk and it contains a mapping between the terms and content. Inverted Index An index if often used in Information Retrieval system...