st: Time to read in data with use/using

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

st: Time to read in data with use/using

sdm1
I'm using Stata 10.1 MP4 with a 64 bit Windows OS and a Citrix Presentation
Server v4.

If I select one variable from a large dataset ('use vara using filename') it
takes Stata 37 minutes to read in the data (vara).

If I save this one variable to a file and then read in this file (vara.dta)
it takes less than 15 seconds to be read in.

I thought that 'use vara using filename' meant that only vara would be read
in.  But if so, why does it take so much longer to read in the same data
when it is extracted from a larger file than when it is in a file on its
own?

Thanks.

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/
Reply | Threaded
Open this post in threaded view
|

Re: st: Time to read in data with use/using

David Kantor
At 09:08 AM 12/2/2008, sdm1 wrote:

>[...]
>
>If I select one variable from a large dataset ('use vara using filename') it
>takes Stata 37 minutes to read in the data (vara).
>
>If I save this one variable to a file and then read in this file (vara.dta)
>it takes less than 15 seconds to be read in.
>
>I thought that 'use vara using filename' meant that only vara would be read
>in.  But if so, why does it take so much longer to read in the same data
>when it is extracted from a larger file than when it is in a file on its
>own?

When it is extracted from a larger file, it still has to read the
entire larger file to get the targetted variable(s). That's a simple
fact about how file storage works.
HTH,
--David

*
*   For searches and help try:
*   http://www.stata.com/help.cgi?search
*   http://www.stata.com/support/statalist/faq
*   http://www.ats.ucla.edu/stat/stata/