SmartView and Large Lists
Wednesday, March 16, 2011
On of the new features i nsmartview really helps with large volumes of data. I want to dig deep into this really important aspect of SmartView and discuss how it used to work and how it now works in the latest release of SmartView.
SmartView is built for speed. Speed in returning data, speed in filtering data, speed in searching for data, speed in sorting and speed in exporting data. By default SmartView loads the data into a local cache so that when you place a filter or sort on the data the processing happens locally and runs really quickly. This is a great feature. The problem that this causes, in the first release of SmartView, is that when you have the maximum records returned set to say 10,000 – but your list contains 900,000 records only the first 10,000 are returned to screen and into the cache. This means that when you filter the list it is only filtering or searching the 10,000 records that are on the screen.
I am the first person to admit this was a limitation that confused a number of users who were looking for data. The initial workaround was to change the max records to a big number like 10 million – but this then slowed things down by returning all records before allowing you to apply a filter.
With The current release 11.0.9 and 10.0.4 we have introduced the concept of a large list. You are able to flag any given list as a Large List which will change the behaviour of this list. A large list will always re query the database to return records that match your filter criteria. Large Lists ensure that you always get the right data while at the same time ensuring smaller lists run at maximum speed.
FAQ’s on this topic: Which lists should be marked as large lists? Any list that is likely to contain more records than your maximum record setting should be flagged as a large list.
Can I mark all my lists as Large Lists? Absolutely you can do this. I would only recommend doing this for lists that will grow rapidly such as transaction lists. The more fixed length lists such as accounts ,customers, items etc will perform better if left as a regular list. Lists such as GL Account Transactions, Sales Transaction, vouchers etc may be best set to be large lists.
How should I use SmartView differently to SmartLists? Something we have found is that many end users select a SmartList list and as soon as it starts returning data they press stop. They then apply their criteria and allow it to run again. This does not work so well with SmartView. One of the big differences between SV and SL is that SV goes and gets all requested records and then displays them at once. SL goes and get record one and displays it, then record 2 and displays and so forth. So if you are asking for 1,000,000 records to be returned SL will return the first record quickly but take a very long time to return all records. SmartView will take a little longer initially but will return all records much faster than SL.
SmartView is built for speed. Speed in returning data, speed in filtering data, speed in searching for data, speed in sorting and speed in exporting data. By default SmartView loads the data into a local cache so that when you place a filter or sort on the data the processing happens locally and runs really quickly. This is a great feature. The problem that this causes, in the first release of SmartView, is that when you have the maximum records returned set to say 10,000 – but your list contains 900,000 records only the first 10,000 are returned to screen and into the cache. This means that when you filter the list it is only filtering or searching the 10,000 records that are on the screen.
I am the first person to admit this was a limitation that confused a number of users who were looking for data. The initial workaround was to change the max records to a big number like 10 million – but this then slowed things down by returning all records before allowing you to apply a filter.
With The current release 11.0.9 and 10.0.4 we have introduced the concept of a large list. You are able to flag any given list as a Large List which will change the behaviour of this list. A large list will always re query the database to return records that match your filter criteria. Large Lists ensure that you always get the right data while at the same time ensuring smaller lists run at maximum speed.
FAQ’s on this topic: Which lists should be marked as large lists? Any list that is likely to contain more records than your maximum record setting should be flagged as a large list.
Can I mark all my lists as Large Lists? Absolutely you can do this. I would only recommend doing this for lists that will grow rapidly such as transaction lists. The more fixed length lists such as accounts ,customers, items etc will perform better if left as a regular list. Lists such as GL Account Transactions, Sales Transaction, vouchers etc may be best set to be large lists.
How should I use SmartView differently to SmartLists? Something we have found is that many end users select a SmartList list and as soon as it starts returning data they press stop. They then apply their criteria and allow it to run again. This does not work so well with SmartView. One of the big differences between SV and SL is that SV goes and gets all requested records and then displays them at once. SL goes and get record one and displays it, then record 2 and displays and so forth. So if you are asking for 1,000,000 records to be returned SL will return the first record quickly but take a very long time to return all records. SmartView will take a little longer initially but will return all records much faster than SL.
