|Mark Breedlove||Sep 18, 2015 12:37 pm|
|Raffaele Palmieri||Sep 19, 2015 10:27 am|
|Mark A. Matienzo||Sep 19, 2015 12:02 pm|
|Sergio Fernández||Sep 22, 2015 12:22 am|
|Mark Breedlove||Sep 23, 2015 3:53 pm|
|Sergio Fernández||Sep 24, 2015 8:47 am|
|Mark Breedlove||Sep 25, 2015 12:30 pm|
|Mark Breedlove||Oct 2, 2015 12:46 pm|
|Sergio Fernández||Oct 8, 2015 4:04 am|
|Subject:||Re: Scaling Marmotta's LDP interface|
|From:||Mark Breedlove (mb...@dp.la)|
|Date:||Sep 25, 2015 12:30:28 pm|
On Thu, Sep 24, 2015 at 10:47 AM, Sergio Fernández <wik...@apache.org> wrote:
From my experience Postgres takes as much as it can, but never risking the system. We normally also use 40% as orientative figure. But with such small server I'd try to give it more memory (6GB?) to see how it behaves.
OK. I'll experiment with that, too, including, likely, an upgrade of the instance type. It will require some scheduling, and will probably take me some number of days to have it done.
Because that, another key aspect that dramatically improves the performance on updates is I/O. Are you using SSD? Actual SSD, I mean, virtual SSDs are a completely different story.
We're using SSD-backed provisioned IOPS Amazon EBS volumes and EBS-optimized instances. That would mean, of course, that they are virtualized. They're rated currently at 12,000 IOPS (Amazon's own metric that corresponds to actual read/write operations in ways that can be difficult to calculate).
We've raised the bandwidth rating of the volumes once already and suppose that we can try that some more, though one encounters a similar problem to increasing the shared buffers: figuring out how much is the right amount, projecting for the growing size of the database and transactions per second. (How far do we have to go?) The problem of one activity achieving satisfactory performance with low transactions per second and two concurrent ones displaying many multiples the amount is what interests me the most at the moment.
I'm not sure I'd want to go to bare-metal hardware yet, just for a database of 1 million ore:Aggregations; seems like we might want to revisit the choice of backend first. I could, however, resize the volumes again and boost the IOPS. I'll have to schedule a time to do that.
Could you point me in the right direction for the indices `cop`, `cp`, and
`literals`? We have `idx_triples_cspo` on `triples` from the Marmotta installation and created one that we called `idx_triples_c` . Are there optional indices that Marmotta doesn't create when it automatically creates the tables the first time you run it?
Well, internally each LDP-R uses its own context. Therefore all indexes that could improve the lookup would help. Try to create also at least cop. [...]
OK, that's on my to-do list and I'll let you know ... :-)