ACE中内存池有没有办法能让他可动态扩展

yoogera

1、我使用ACE_Cached_Allocator申请一个nsize大小的空间，但是在实际使用中档需要的内存数量大于nsize的时候，就返回一个null。这里我想如果在内存大于nsize的时候系统能自动将内存池的大小扩展，然后还是能正常的分配一个内存给我，有直接能用的类吗？我试验了几个ACe的内存池都是在初始化时指定的nsize，就是他的最大大小，超过的话就返回null。

2、今天进行了一下测试，发现似乎使用ace内存池分配内存的效率还没有直接使用new来看快。大家有这方面的测试吗？（我使用的是ace_cached_allocator，初始大小10000，每个chunk = 300，分配10000次）

modern

兄弟，你运气真好，刚好昨天我把这段代码改完！
有两种办法，1直接修改ACE_Cached_Allocator的源代码。
2.像我下面提供的办法一样将ACE_Cached_Allocator扣出来，改完了放在自己的命名空间里。
这样也不会与原来的代码冲突了。

别害怕，看蓝色字的部分就成了。
注意中间有一处delete必须被注释掉，否则会重复delete导致崩溃
namespace MY
{
template <class T, class ACE_LOCK>
class ACE_Locked_Free_List_Ex :public ACE_Locked_Free_List<T,ACE_LOCK>
{
public:
ACE_Locked_Free_List_Ex (int mode = ACE_FREE_LIST_WITH_POOL,
  size_t prealloc = ACE_DEFAULT_FREE_LIST_PREALLOC,
  size_t lwm = ACE_DEFAULT_FREE_LIST_LWM,
  size_t hwm = ACE_DEFAULT_FREE_LIST_HWM,
  size_t inc = ACE_DEFAULT_FREE_LIST_INC):
  ACE_Locked_Free_List<T,ACE_LOCK>(mode,prealloc,lwm,hwm,inc)
{
}
virtual void alloc (size_t n,size_t nObjSize)
{
  for (; n > 0; n--)
  {
   char *ta = new char[nObjSize];
   T *temp = 0;
   temp = new (ta) T;
   //ACE_NEW (temp, T);
   temp->set_next (this->free_list_);
   this->free_list_ = temp;
   this->size_++;
  }
}
virtual T *remove (size_t nObjSize)
{
  ACE_MT (ACE_GUARD_RETURN (ACE_LOCK, ace_mon, this->mutex_, 0));
  // If we are at the low water mark, add some nodes
  if (this->mode_ != ACE_PURE_FREE_LIST && this->size_ <= this->lwm_)
   this->alloc (this->inc_,nObjSize);
  // Remove a node
  T *temp = this->free_list_;
  if (temp != 0)
  {
   this->free_list_ = this->free_list_->get_next ();
   this->size_--;
  }
  return temp;
}
};

template <class T, class ACE_LOCK>
class ACE_Cached_Allocator : public ACE_New_Allocator
{
public:
  /// Create a cached memory pool with @a n_chunks chunks
  /// each with sizeof (TYPE) size.
  ACE_Cached_Allocator (size_t n_chunks,const int nPoolType = ACE_FREE_LIST_WITH_POOL);

  /// Clear things up.
  ~ACE_Cached_Allocator (void);

  /**
   * Get a chunk of memory from free list cache.  Note that @a nbytes is
   * only checked to make sure that it's less or equal to sizeof T, and is
   * otherwise ignored since @c malloc() always returns a pointer to an
   * item of sizeof (T).
   */
  void *malloc (size_t nbytes = sizeof (T));

  /**
   * Get a chunk of memory from free list cache, giving them
   * @a initial_value.  Note that @a nbytes is only checked to make sure
   * that it's less or equal to sizeof T, and is otherwise ignored since
   * calloc() always returns a pointer to an item of sizeof (T).
   */
  virtual void *calloc (size_t nbytes,
                        char initial_value = '\0');

  /// This method is a no-op and just returns 0 since the free list
  /// only works with fixed sized entities.
  virtual void *calloc (size_t n_elem,
                        size_t elem_size,
                        char initial_value = '\0');

  /// Return a chunk of memory back to free list cache.
  void free (void *);

  /// Return the number of chunks available in the cache.
  size_t pool_depth (void);

private:
  /// Remember how we allocate the memory in the first place so
  /// we can clear things up later.
  char *pool_;

  /// Maintain a cached memory free list.
  ACE_Locked_Free_List_Ex<ACE_Cached_Mem_Pool_Node<T>, ACE_LOCK> free_list_;
};

template <class T, class ACE_LOCK>
ACE_Cached_Allocator<T, ACE_LOCK>::ACE_Cached_Allocator (size_t n_chunks,const int nPoolType)
: pool_ (0),
free_list_ (nPoolType)
{
        // To maintain alignment requirements, make sure that each element
        // inserted into the free list is aligned properly for the platform.
        // Since the memory is allocated as a char[], the compiler won't help.
        // To make sure enough room is allocated, round up the size so that
        // each element starts aligned.
        //
        // NOTE - this would probably be easier by defining pool_ as a pointer
        // to T and allocating an array of them (the compiler would probably
        // take care of the alignment for us), but then the ACE_NEW below would
        // require a default constructor on T - a requirement that is not in
        // previous versions of ACE
        size_t chunk_size = sizeof (T);
        chunk_size = ACE_MALLOC_ROUNDUP (chunk_size, ACE_MALLOC_ALIGN);
        ACE_NEW (this->pool_,
                char[n_chunks * chunk_size]);

        for (size_t c = 0;
                c < n_chunks;
                c++)
        {
                void* placement = this->pool_ + c * chunk_size;
                this->free_list_.add (new (placement) ACE_Cached_Mem_Pool_Node<T>);
        }
        // Put into free list using placement contructor, no real memory
        // allocation in the above <new>.
}

template <class T, class ACE_LOCK>
ACE_Cached_Allocator<T, ACE_LOCK>::~ACE_Cached_Allocator (void)
{
       //delete [] this->pool_;
}

template <class T, class ACE_LOCK> void *
ACE_Cached_Allocator<T, ACE_LOCK>::malloc (size_t nbytes)
{
        // Check if size requested fits within pre-determined size.
        if (nbytes > sizeof (T))
                return 0;

        // addr() call is really not absolutely necessary because of the way
        // ACE_Cached_Mem_Pool_Node's internal structure arranged.
        return this->free_list_.remove (obj_size)->addr ();
}

template <class T, class ACE_LOCK> void *
ACE_Cached_Allocator<T, ACE_LOCK>::calloc (size_t nbytes,
                                                                                   char initial_value)
{
        // Check if size requested fits within pre-determined size.
        if (nbytes > sizeof (T))
                return 0;

        // addr() call is really not absolutely necessary because of the way
        // ACE_Cached_Mem_Pool_Node's internal structure arranged.
        void *ptr = this->free_list_.remove (obj_size)->addr ();
        if (ptr != 0)
                ACE_OS::memset (ptr, initial_value, sizeof (T));
        return ptr;
}

template <class T, class ACE_LOCK> void *
ACE_Cached_Allocator<T, ACE_LOCK>::calloc (size_t,
                                                                                   size_t,
                                                                                   char)
{
        ACE_NOTSUP_RETURN (0);
}

template <class T, class ACE_LOCK> void
ACE_Cached_Allocator<T, ACE_LOCK>::free (void * ptr)
{
        if (ptr != 0)
                this->free_list_.add ((ACE_Cached_Mem_Pool_Node<T> *) ptr) ;
}
}

用法
typedef char MEMORY_DATA[1024];
typedef MY::ACE_Cached_Allocator<MEMORY_DATA,ACE_SYNCH_MUTEX> MyDataAlloc;
MyDataAllocEx aa(100);

[ 本帖最后由 modern 于 2010-7-8 13:03 编辑 ]

modern

我测试的结果n_chunks = 20k，
相对于OSnew，delete，在O2优化的情况
如果内存分配出来后有初始化的话，大概有1:1.5（pool：os）的性能差异。

[ 本帖最后由 modern 于 2010-7-6 22:25 编辑 ]

yoogera

的确是个办法，我也比较奇怪，为什么ace不把那个参数直接暴露出来呢～～～，做成了个死的

steven99ca

的确是个办法，我也比较奇怪，为什么ace不把那个参数直接暴露出来呢～～～，做成了个死的 ...
yoogera 发表于 2010-7-6 20:48

    1
ACE_Cached_Allocator的设计没有问题，这是典型的基本内存池，避免碎片，定时存取。最安全和高效的用法是一次分配足够大，

2
如果实在要多次分配，你应该使用现有的list, 重新实现你自己的allocator.
而不是修改list本身。嫁接本身会产生新的问题。

freebird92

根据 modern 的一些想法， 我改了一个可扩展有ACE_Cached_Allocator;  代码如下， 有好点子的兄弟们不妨帮我找出出主意，多谢 。

template <class T, class ACE_LOCK>
class ACE_Cached_Allocator_Ex : public ACE_New_Allocator
{
public:
        /// Create a cached memory pool with @a n_chunks chunks
        /// each with sizeof (TYPE) size.
        ACE_Cached_Allocator_Ex (size_t n_chunks);

        /// Clear things up.
        ~ACE_Cached_Allocator_Ex (void);

        /**
        * Get a chunk of memory from free list cache.  Note that @a nbytes is
        * only checked to make sure that it's less or equal to sizeof T, and is
        * otherwise ignored since @c malloc() always returns a pointer to an
        * item of sizeof (T).
        */
        void *malloc (size_t nbytes = sizeof (T));

        /**
        * Get a chunk of memory from free list cache, giving them
        * @a initial_value.  Note that @a nbytes is only checked to make sure
        * that it's less or equal to sizeof T, and is otherwise ignored since
        * calloc() always returns a pointer to an item of sizeof (T).
        */
        virtual void *calloc (size_t nbytes,
                char initial_value = '\0');

        /// This method is a no-op and just returns 0 since the free list
        /// only works with fixed sized entities.
        virtual void *calloc (size_t n_elem,
                size_t elem_size,
                char initial_value = '\0');

        /// Return a chunk of memory back to free list cache.
        void free (void *);

        /// Return the number of chunks available in the cache.
        size_t pool_depth (void);

protected:
        /// Add a new pool;       
        void increase_pool();

private:
        /// Remember how we allocate the memory in the first place so
        /// we can clear things up later.
        char **pool_;

        /// the number of the pools we have totally allocated..
        int        number_of_pool;

        /// Synchronization variable for pool increasing..
        ACE_LOCK mutex_;        /// Maintain a cached memory free list.
        ACE_Locked_Free_List<ACE_Cached_Mem_Pool_Node<T>, ACE_LOCK> free_list_;
};
///////////////////////////////////////////////////////////////////////////
///////////////////////CPP part////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////
template <class T, class ACE_LOCK>
ACE_Cached_Allocator_Ex<T, ACE_LOCK>::ACE_Cached_Allocator_Ex (size_t n_chunks)
: pool_ (0),
free_list_ (ACE_PURE_FREE_LIST)
{
        // To maintain alignment requirements, make sure that each element
        // inserted into the free list is aligned properly for the platform.
        // Since the memory is allocated as a char[], the compiler won't help.
        // To make sure enough room is allocated, round up the size so that
        // each element starts aligned.
        //
        // NOTE - this would probably be easier by defining pool_ as a pointer
        // to T and allocating an array of them (the compiler would probably
        // take care of the alignment for us), but then the ACE_NEW below would
        // require a default constructor on T - a requirement that is not in
        // previous versions of ACE
        size_t chunk_size = sizeof (T);
        chunk_size = ACE_MALLOC_ROUNDUP (chunk_size, ACE_MALLOC_ALIGN);

        // here allocate the first memory pool
        number_of_pool = 1;

        char* ptr_pool;
        ACE_NEW (ptr_pool,
                char[n_chunks * chunk_size]);

        char** pools;
        ACE_NEW(pools,
                char*[number_of_pool]);

        pools[number_of_pool - 1] = ptr_pool;
        this->pool_ = pools;

        for (size_t c = 0;
                c < n_chunks;
                c++)
        {
                void* placement = ptr_pool + c * chunk_size;
                this->free_list_.add (new (placement) ACE_Cached_Mem_Pool_Node<T>);
        }
        // Put into free list using placement contructor, no real memory
        // allocation in the above <new>.
}

template <class T, class ACE_LOCK>
ACE_Cached_Allocator_Ex<T, ACE_LOCK>::~ACE_Cached_Allocator_Ex (void)
{

        for (size_t c=0;
                c<number_of_pool;
                c++)
        {
                delete[] this->pool_[c];
        }

        delete [] this->pool_;
}

template <class T, class ACE_LOCK> void *
ACE_Cached_Allocator_Ex<T, ACE_LOCK>::malloc (size_t nbytes)
{
        // Check if size requested fits within pre-determined size.
        if (nbytes > sizeof (T))
                return 0;

        // addr() call is really not absolutely necessary because of the way
        // ACE_Cached_Mem_Pool_Node's internal structure arranged.
        void* tmp = this->free_list_.remove ()->addr ();
        if (!tmp)
        {
                increase_pool();
                tmp = this->free_list_.remove ()->addr ();
        }

        return tmp;
}
template <class T, class ACE_LOCK> void
ACE_Cached_Allocator_Ex<T, ACE_LOCK>::increase_pool()
{
        ACE_MT (ACE_GUARD (ACE_LOCK, ace_mon, this->mutex_));
        //        expand the memory pool when there is not enough memory chunks.
        size_t chunk_size = sizeof (T);
        chunk_size = ACE_MALLOC_ROUNDUP (chunk_size, ACE_MALLOC_ALIGN);

        // here allocate the extra memory pool pointer
        number_of_pool ++;

        char** pools;
        ACE_NEW(pools,
                char*[number_of_pool]);

        for (int i=0; i<number_of_pool; i++)
        {
                pools = this->pool_;
        }

        delete[] this->pool_;
        this->pool_ = pools;

        // here allocate the extra memory pool
        char* ptr_pool;
        ACE_NEW (ptr_pool,
                char[ACE_DEFAULT_FREE_LIST_INC * chunk_size]);

        pools[number_of_pool - 1] = ptr_pool;
        for (size_t c = 0;
                c < ACE_DEFAULT_FREE_LIST_INC;
                c++)
        {
                void* placement = ptr_pool + c * chunk_size;
                this->free_list_.add (new (placement) ACE_Cached_Mem_Pool_Node<T>);
        }
}
template <class T, class ACE_LOCK> void *
ACE_Cached_Allocator_Ex<T, ACE_LOCK>::calloc (size_t nbytes,
                                                                                          char initial_value)
{
        // Check if size requested fits within pre-determined size.
        if (nbytes > sizeof (T))
                return 0;

        // addr() call is really not absolutely necessary because of the way
        // ACE_Cached_Mem_Pool_Node's internal structure arranged.
        void *ptr = this->free_list_.remove ()->addr ();
        if (ptr != 0)
                ACE_OS::memset (ptr, initial_value, sizeof (T));
        return ptr;
}

template <class T, class ACE_LOCK> void *
ACE_Cached_Allocator_Ex<T, ACE_LOCK>::calloc (size_t,
                                                                                   size_t,
                                                                                   char)
{
        ACE_NOTSUP_RETURN (0);
}

template <class T, class ACE_LOCK> void
ACE_Cached_Allocator_Ex<T, ACE_LOCK>::free (void * ptr)
{
        if (ptr != 0)
                this->free_list_.add ((ACE_Cached_Mem_Pool_Node<T> *) ptr) ;
}


template <class T, class ACE_LOCK> ACE_INLINE size_t
ACE_Cached_Allocator_Ex<T, ACE_LOCK>::pool_depth (void)
{
        return this->free_list_.size ();
}

freeeyes

我有时间测试一下。

z_kris

1，看代码
template <class T, class ACE_LOCK> void *
ACE_Cached_Allocator<T, ACE_LOCK>::malloc (size_t nbytes)
{
  // Check if size requested fits within pre-determined size.
  if (nbytes > sizeof (T))
    return 0;

  // addr() call is really not absolutely necessary because of the way
  // ACE_Cached_Mem_Pool_Node's internal structure arranged.
  return this->free_list_.remove ()->addr ();
}

z_kris

第一个问题，其实ACE_Cached_Allocator会动态扩展，但是只会扩展固定大小的chunk，因为他只是个cache的分配器，所以当你请求的大小超过sizeof(T)时会返回NULL。如果你需要更大的内存块，你应该定义一个ACE_Cached_Allocator<T2,***_Mutex>，sizeof(T2)大于你请求的大小就行了。
第二个问题，我在linux下测试是比malloc的效率要高。