lundi 30 mai 2016

Design considerations for high volume search

I am working to come up with solution for following problem scenario: Say, an organization offers General & specialized deals for its customers. General deals apply to ALL customer but specialized deals only for those who meet defined criteria (e.g.; customer of Gold loyalty class, customer with age between 18-26 etc.). Now requirement is that when a customer logs onto organization's portal, he/she is offered the deals that are applicable specifically to him (so general + specialized ones of which Customer meets conditions). Trouble is there are hundreds of possible deals & few million of customer. So, for each specific customer - it will mean searching through hundred of deals and applying various match criteria logic. Imagine situation when there can be hundreds of concurrent users. System is web-based with possibility of mainframe data store, application logic. What solution do you suggest to handle such huge volume of search logic to be executed? Possible options that I see: (1) On mainframe, have a batch application do the heavy processing of identifying what deals apply to which customer and store the deal-id, customer-id as a result in RDBMS. This might mean, that if on an average 4 deals apply and there are 2 million customer then there will be roughly 8 million rows of data. This will however, make life easy for web-portal to search applicable deals with just specifying the customer-id. Of course, Some performance tuning will be required to speed-up response time for the (single) query. But on high level, it will work. (2) Web-portal application might check eligibility of all deals for entered customer-id, one after the another. This will mean, that if there are X nr of deals then for each customer login, application will (a) Get all relevant customer details from database A, (b) Match retrieved customer details with X nr of deals criteria, to identify the matching ones that can finally be shown to customer. This approach however will mean execution of tens of complex queries on different databases, which ultimately will slow-down response time for the customer.

Do you see other options and better solution approach?! Please suggest if there is any known design patter/ approach to handle such problems.

Thanks, GG

Aucun commentaire:

Enregistrer un commentaire