Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Speed of adding document to azure directory. #2

Open
pe4aiko opened this issue Mar 26, 2013 · 1 comment
Open

Speed of adding document to azure directory. #2

pe4aiko opened this issue Mar 26, 2013 · 1 comment

Comments

@pe4aiko
Copy link

pe4aiko commented Mar 26, 2013

Hi. I have some code:
var cloudAccount = CloudStorageAccount.Parse("someAccount");
var cacheDirectory = new RAMDirectory();
var azureDirectory = new AzureDirectory(cloudAccount, "fulltextcloudindex", cacheDirectory);
Analyzer analyzer = new StandardAnalyzer(Version.LUCENE_30);

var indexWriter = new IndexWriter(azureDirectory, analyzer, true, IndexWriter.MaxFieldLength.UNLIMITED);
var luceneDataProvider = new LuceneDataProvider(azureDirectory, Version.LUCENE_30, IndexWriter);

using (var sessionArticle = LuceneDataProvider.OpenSession())
{
sessionArticle.Add(entity);
}

When I try to step out from using.. it gets about 10 seconds. It's really slow. Or maybe I need to do it in another way?

The output:
COMPRESSED 201 -> 168 83.58209% to _2g.fdt
PUT 168 bytes to _2g.fdt in cloud
CLOSED WRITESTREAM _2g.fdt
COMPRESSED 12 -> 9 75% to _2g.fdx
PUT 9 bytes to _2g.fdx in cloud
CLOSED WRITESTREAM _2g.fdx
COMPRESSED 348 -> 230 66.09196% to _2g.tis
PUT 230 bytes to _2g.tis in cloud
CLOSED WRITESTREAM _2g.tis
COMPRESSED 35 -> 26 74.28571% to _2g.tii
PUT 26 bytes to _2g.tii in cloud
CLOSED WRITESTREAM _2g.tii
COMPRESSED 28 -> 12 42.85714% to _2g.frq
AzureLock:Renew(write.lock : fe28a1f6-2597-4383-85f8-5efe27c23534
PUT 12 bytes to _2g.frq in cloud
CLOSED WRITESTREAM _2g.frq
COMPRESSED 20 -> 17 85% to _2g.prx
PUT 17 bytes to _2g.prx in cloud
CLOSED WRITESTREAM _2g.prx
COMPRESSED 16 -> 12 75% to _2g.nrm
PUT 12 bytes to _2g.nrm in cloud
CLOSED WRITESTREAM _2g.nrm
PUT 155 bytes to _2g.fnm in cloud
CLOSED WRITESTREAM _2g.fnm
opening _2g.fdt
Using cached file for _2g.fdt
CLOSED READSTREAM local _2g.fdt
opening _2g.fdx
Using cached file for _2g.fdx
CLOSED READSTREAM local _2g.fdx
opening _2g.tis
Using cached file for _2g.tis
CLOSED READSTREAM local _2g.tis
opening _2g.tii
Using cached file for _2g.tii
CLOSED READSTREAM local _2g.tii
opening _2g.frq
Using cached file for _2g.frq
CLOSED READSTREAM local _2g.frq
opening _2g.prx
Using cached file for _2g.prx
CLOSED READSTREAM local _2g.prx
opening _2g.nrm
Using cached file for _2g.nrm
CLOSED READSTREAM local _2g.nrm
opening _2g.fnm
Using cached file for _2g.fnm
CLOSED READSTREAM local _2g.fnm
COMPRESSED 944 -> 528 55.93221% to _2g.cfs
PUT 528 bytes to _2g.cfs in cloud
CLOSED WRITESTREAM _2g.cfs
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2g.fnm
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2g.frq
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2g.prx
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2g.tis
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2g.tii
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2g.nrm
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2g.fdx
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2g.fdt
opening _2g.cfs
Using cached file for _2g.cfs
Creating clone for _2g.cfs
CLOSED READSTREAM local
Creating clone for _2g.cfs
Creating clone for _2g.cfs
CLOSED READSTREAM local
Creating clone for _2g.cfs
Creating clone for _2g.cfs
Creating clone for _2g.cfs
Creating clone for
CLOSED READSTREAM local
Creating clone for
Creating clone for
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local _2g.cfs
AzureLock:Renew(write.lock : fe28a1f6-2597-4383-85f8-5efe27c23534
PUT 212 bytes to segments_2x in cloud
CLOSED WRITESTREAM segments_2x
PUT 20 bytes to segments.gen in cloud
CLOSED WRITESTREAM segments.gen
DELETE https://company.blob.core.windows.net/fulltextcloudindex/segments_2w
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2d.cfs
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2d_1.del
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2e.cfs
DELETE https://company.blob.core.windows.net/fulltextcloudindex/_2f.cfs
opening segments.gen
Using cached file for segments.gen
CLOSED READSTREAM local segments.gen
opening segments_2x
Using cached file for segments_2x
CLOSED READSTREAM local segments_2x
opening _2g.cfs
Using cached file for _2g.cfs
Creating clone for _2g.cfs
CLOSED READSTREAM local
Creating clone for _2g.cfs
Creating clone for _2g.cfs
CLOSED READSTREAM local
Creating clone for _2g.cfs
Creating clone for _2g.cfs
Creating clone for _2g.cfs
Creating clone for _2g.cfs
Creating clone for
Creating clone for
Creating clone for _2g.cfs
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local _2d.cfs
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local _2e.cfs
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local
CLOSED READSTREAM local _2f.cfs

@richorama
Copy link
Member

Sorry, I just ported the code to the new storage library, so I don't really have the authority to talk about performance expectations.

Just out of interest, what size blobs did it create? Are you running this inside the same data centre as the storage account?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants