Latest Z Technologies

what is the best language to develop a search engine?and what latest technologies can i use?

like java,perl,python etc

Public Comments

1. it is asp
2. Modern search engines require massive databases (such as the ones used by yahoo and google). Currently, only C or C++ running on UNIX have enough power to shift through the amount of data needed. Of course, if you're plans are less ambitious, you can easily use any of the languages you listed, and any OS.
3. I agree with Jonathan's excellent answer. I would only add one the following: Search engine software will be dealing with strings and string manipulation in a big way, so you might want to think about native language string support. Native string support in C, for example, is weak at best, while C++ provides powerful string classes. Of the languages you mention, Perl has very good string support, suitable for a smaller scale project. My preference would be Java, however, for anything significant that needs room to grow. Consider a Java servlet running under Apache Tomcat with a MySQL backend. A modular approach is important, if you expect the project to scale into the mainstream. At some point, you might want to replace MySQL with Oracle, or the search engine backend with C++, while maintaining a Java servlet at the front-end, for example. Added Note: Don't let anyone tell you there is one "best" language. That would be more like religion than technology. If you want the fastest, assembly language performs the "best" by far, but you'd have to write code for a lifetime. Always select the technologies most appropriate for your needs, while considering availability of skills and availability of resources. Added Note 2: Google's technology is based on distributed computing -- massive numbers of PCs working in parallel. This architecture is not well-suited to a RDBMS. Google is one search engine, not every search engine. Added Note 3: Avoid Windows for anything not on the desktop. Now that's religion!
4. The best technology would be python. Python is made for search technology. For people who responded above... Databases are not used for search engines. They are SLOW! Google DOES not run on MySQL it runs on its own database with PERL + PYTHON. But to develop for yourself, I would recommend ASP.NET with Micrsoft SQL Server 2005. To develop Extensive Applications. They are the best. All the big crucial companies depend on their service. But if your just doing a simple application you can use JAVA with derby or with any of the dbms outthere. PERL reads FILES QUICKlY PYTHON SEARCHES quickest the combination of both is GOOGLE LIKE! :)
5. Hi. The term "best language" is a bit of a misleading question since it really depends on a lot of your own personal factors. For instance, are you most comfortable in one language over another? Does your organization/site use a particular language as a standard? Your answers to those questions might affect how you interpret the "best language." But I digress, from my own work-related experimentation, we've settled on using Lucene, a Java-based indexer/searcher (which, by the way, has an existing web search engine project called Nutch). It has a very Google-like query syntax for searching (see reference #3, also see reference #4 for a general feature set). We're actually primarily a Perl-based organization, so we've added another layer to the mix. We use the Lucene Web Service Java Servlet which allows us to use the supplied Perl client to index and search while still giving us the power and speed of Java Lucene. Hope that helps.
6. I think that there is no such a thing like a "best language", since what is best for one user may not be the best for another. The "best" will depend on your previous experience and will to learn. There are things like appropriate features for a particular task. In this case, you need to produce fast code since you will have to process a lot on information. Interpreted languages usually are not enough for this task. You could use PHP or Python to serve the web page of a search engine, but I wouldn't use it to index the database. I would use C for that. You could use a scripting language for link retrieval (spider function), since the speed is limited by the net speed and not for the programming language. Here a sample search engine written in Python: www.taniquetil.com.ar/facundo/bdvfiles/python-baterias-1.0.sxi
7. Optimality differs according to context. As is said above, no language is necessarily the best language as each one has its strengths and weaknesses. You need to piece together the information and compare your own set of skills and knowledge against what is already out there. There are plenty of examples and resources to help you get started. Don't adapt to whatever language is perceived as the best, if you are starting from scratch it is a good idea to build with what you are familiar with first. You can always convert what you've built over to another language later once you know the in's and outs. Good luck, and let me know how it turns out.