Assert `verify_maplen_equivalent_to_mapaddr() == true' failed.

[pramalhe]

Hi,
After the fix we see this assert in some situations. This is not blocking us so no rush.
If I manage to find some time next week I'll try to make a unit test that reproduces this issue and do a pull request with it, but no promises ;)

Meanwhile, here is the relevant part of the stack decode:

bench: /vagrant/mnemosyne-gcc/usermode/library/pmalloc/include/alps/src/layers/extentmap.cc:176: void alps::ExtentMap::insert(const alps::ExtentInterval&): A
ssertion `verify_maplen_equivalent_to_mapaddr() == true' failed.

Program received signal SIGABRT, Aborted.
0x00007ffff59b9c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
56 ../nptl/sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0 0x00007ffff59b9c37 in __GI_raise (sig=sig@entry=6) at ../nptl/sysdeps/unix/sysv/linux/raise.c:56
#1 0x00007ffff59bd028 in __GI_abort () at abort.c:89
#2 0x00007ffff59b2bf6 in __assert_fail_base (fmt=0x7ffff5b07018 "%s%s%s:%u: %s%sAssertion `%s' failed.\n%n",
assertion=assertion@entry=0x7ffff6565d00 "verify_maplen_equivalent_to_mapaddr() == true",
file=file@entry=0x7ffff6565ca8 "/vagrant/mnemosyne-gcc/usermode/library/pmalloc/include/alps/src/layers/extentmap.cc", line=line@entry=176,
function=function@entry=0x7ffff6566060 <alps::ExtentMap::insert(alps::ExtentInterval const&)::PRETTY_FUNCTION> "void alps::ExtentMap::insert(const al
ps::ExtentInterval&)") at assert.c:92
#3 0x00007ffff59b2ca2 in __GI___assert_fail (assertion=0x7ffff6565d00 "verify_maplen_equivalent_to_mapaddr() == true",
file=0x7ffff6565ca8 "/vagrant/mnemosyne-gcc/usermode/library/pmalloc/include/alps/src/layers/extentmap.cc", line=176,
function=0x7ffff6566060 <alps::ExtentMap::insert(alps::ExtentInterval const&)::PRETTY_FUNCTION> "void alps::ExtentMap::insert(const alps::ExtentInter
val&)") at assert.c:101
#4 0x00007ffff6509194 in alps::ExtentMap::insert(alps::ExtentInterval const&) () from library/pmalloc/include/alps/build/src/libalps.so
#5 0x00007ffff679ddae in alps::FreeSpaceMap<Context, alps::TPtr>::free_extent (this=0x68da50, ctx=..., ex=...)
at library/pmalloc/include/alps/include/alps/layers/bits/freespacemap.hh:60
#6 0x00007ffff679d11f in alps::ExtentHeap<Context, alps::TPtr, alps::PPtr>::free_extent (this=0x68da20, ctx=..., ex=...)
at library/pmalloc/include/alps/include/alps/layers/extentheap.hh:189
#7 0x00007ffff679b884 in alps::ExtentHeap<Context, alps::TPtr, alps::PPtr>::free_extent (this=0x68da20, ctx=..., ptr=...)
at library/pmalloc/include/alps/include/alps/layers/extentheap.hh:198
#8 0x00007ffff6799ea6 in alps::ExtentHeap<Context, alps::TPtr, alps::PPtr>::free (this=0x68da20, ctx=..., ptr=...)
at library/pmalloc/include/alps/include/alps/layers/extentheap.hh:223
#9 0x00007ffff6798ecd in alps::HybridHeap<Context, alps::TPtr, alps::PPtr, alps::SlabHeap<Context, alps::TPtr, alps::PPtr>, alps::ExtentHeap<Context, alps::
TPtr, alps::PPtr> >::free (this=0x68dd80, ctx=..., ptr=...) at library/pmalloc/include/alps/include/alps/layers/hybridheap.hh:59
#10 0x00007ffff6797b23 in ThreadHeap::pfree_commit (this=0x68d940, ptr=0x1000241519d0) at build/library/pmalloc/src/heap.cc:100
#11 0x00007ffff679efaf in mtm_pfree_commit (ptr=0x1000241519d0) at build/library/pmalloc/src/wrapper.cc:87
#12 0x00007ffff73d08fb in mtm_useraction_list_run (list=0x68dd60, reverse=0) at build/library/mtm/src/useraction.c:115
#13 0x00007ffff73c5cf5 in trycommit_transaction (tx=0x689c00, enable_isolation=1) at build/library/mtm/include/mode/pwb-common/beginend-bits.h:436
#14 0x00007ffff73c5f00 in mtm_pwbetl_commitTransaction (tx=0x689c00, loc=0x0) at build/library/mtm/src/mode/pwbetl/beginend.c:108
#15 0x00007ffff73c0382 in _ITM_commitTransaction () at build/library/mtm/src/gcc-abi.c:234

[hvolos]

Does this happen with the bigalloc example/test or another program?
Could you please try running with debug-logging on and see whether you could get a trace? Turning logging on is a bit inconvenient at the moment as it requires removing (or commenting) the call to init_log in Heap::init (mnemosyne-gcc/usermode/library/pmalloc/src/heap.cc).

[pramalhe]

It happens in our of our stress tests. We're using it for our own STM/PTM and subjected Mnemosyne to the same one ;)
The test does something like this:

1. Create a persistent array (size 512 entries, but the more the better) of pointers to Foo;
2. Initialize each entry of the array with a difference Foo instance, allocated with pmalloc();
3. Do transactions for 20 seconds in a loop where each transaction does:
   a) Pick a random entry of the persistent array;
   b) De-reference some of the contents of Foo and make sure it's still consistent (test some invariant);
   c) Allocate a new Foo instance and replace the old pointer in the array with a pointer to the newly allocated Foo;
   d) Free the old Foo instance using pfree()

Yes this does a lot of memory fragmentation, but it's a great way to test the memory allocator. The more threads you put concurrently doing this (for the same array) the better.
It's also nice to run it under Address Sanitizer or valgrind to make sure there are no leaks (in volatile memory) or overflows.

[pramalhe]

Hello,
I can confirm that the issue with the large array was fixed by increasing the RW_SET_SIZE in the config 👍

Unfortunately, this particular issue is still there.
I've enabled the test that reproduces this issue on my laptop, hopefully you will be able to repro it too (it's on the latest pull request).

Once again thanks for the help,
Pedro

[hvolos]

Yes, I can reproduce this. I'll try to take a look.

[hvolos]

My previous fix to issue #9 was apparently wrong, triggering this assertion. I pushed a new fix, which hopefully put this issue to rest. With the new fix, I am able to pass the rbench test.

[hvolos]

Looks like the new fix triggers issue #9.

[hvolos]

Latest commit bb59f5c fixes this issue and issue #9

[hvolos]

With the latest commit bb59f5c I am now able to pass both the rbench and bigalloc test.

[pramalhe]

Yep, I can confirm, this is working :)
Thanks for the help!