in Education by
While training a tensorflow seq2seq model I see the following messages : W tensorflow/core/common_runtime/gpu/pool_allocator.cc:227] PoolAllocator: After 0 get requests, put_count=2362 evicted_count=2000 eviction_rate=0.84674 and unsatisfied allocation rate=-nan W tensorflow/core/common_runtime/gpu/pool_allocator.cc:227] PoolAllocator: After 38 get requests, put_count=5436 evicted_count=5000 eviction_rate=0.919794 and unsatisfied allocation rate=0 What does it mean, does it mean I am having some resource allocation issues? Am running on Titan X 3500+ CUDA ,12 GB GPU Select the correct answer from above options

1 Answer

0 votes
by
 
Best answer
There are multiple memory allocators in TensorFlow, so that memory can be used in different ways. Their behavior has some adaptive aspects. For GPU users, there is a PoolAllocator for CPU memory that is pre-registered with the GPU for fast DMA. A tensor that is expected to be transferred from CPU to GPU, e.g., will be allocated from this pool. The main working concept of PoolAllocators is to decompose the cost of calling a more expensive underlying allocator by keeping around a pool of allocated then freed chunks that are eligible for immediate reuse. Their default approach is to grow slowly until the removal rate drops below some constant. The removal rate is the proportion of free calls where we return an unused part from the pool to the underlying pool in order not to exceed the size limit.) In the log messages above, you see "Raising pool_size_limit_" lines that show the pool size growing. If your program actually has a steady state behavior with a maximum size collection of chunks it needs, the pool will grow to accommodate it, and then grow no more. It behaves this way rather than simply retaining all chunks ever allocated so that sizes needed only rarely, or only during program startup, are less likely to be retained in the pool. These messages should only be a reason for concern if you run out of memory. In such a case the log messages may help diagnose the problem. The peak execution speed may only be attained after the memory pools have grown to the proper size.

Related questions

0 votes
    While training a tensorflow seq2seq model I see the following messages : W tensorflow/core/common_runtime/gpu/pool_allocator ... GB GPU Select the correct answer from above options...
asked Feb 5, 2022 in Education by JackTerrance
0 votes
    Classification problems, such as logistic regression or multinomial logistic regression, optimize a cross-entropy loss. ... jungle. Select the correct answer from above options...
asked Feb 4, 2022 in Education by JackTerrance
0 votes
    I am trying to groupby a column and compute value counts on another column. import pandas as pd dftest = pd. ... Amt, already exists Select the correct answer from above options...
asked Feb 1, 2022 in Education by JackTerrance
0 votes
    I have just built my first model using Keras and this is the output. It looks like the standard output you get ... - loss: 0.1928 Select the correct answer from above options...
asked Feb 1, 2022 in Education by JackTerrance
0 votes
    I'm starting with input data like this df1 = pandas.DataFrame( { "Name" : ["Alice", "Bob", "Mallory", ... Any hints would be welcome. Select the correct answer from above options...
asked Jan 28, 2022 in Education by JackTerrance
0 votes
    Suppose I have a Tensorflow tensor. How do I get the dimensions (shape) of the tensor as integer values? I ... 'Dimension' instead. Select the correct answer from above options...
asked Feb 8, 2022 in Education by JackTerrance
0 votes
    The classifiers in machine learning packages like liblinear and nltk offer a method show_most_informative_features(), which ... lot! Select the correct answer from above options...
asked Feb 4, 2022 in Education by JackTerrance
0 votes
    How can I extract the decision path as a textual list from a trained tree in a decision tree ? Something similar to ... then class='A' Select the correct answer from above options...
asked Jan 22, 2022 in Education by JackTerrance
0 votes
    Today I'm trying to learn something about K-means. I Have understood the algorithm and I know how it works. Now I ... a lot of time? Select the correct answer from above options...
asked Jan 29, 2022 in Education by JackTerrance
0 votes
    I can't decipher however the sklearn.pipeline.Pipeline works precisely. Some explanation in this documentation. What ... estimator)]) Select the correct answer from above options...
asked Jan 22, 2022 in Education by JackTerrance
0 votes
    How can I save a trained Naive Bayes classifier to a disk and use it for predicting data? Select the correct answer from above options...
asked Jan 22, 2022 in Education by JackTerrance
0 votes
    I am training on 970 samples and validating on 243 samples. How big should batch size and number of epochs be ... on data input size? Select the correct answer from above options...
asked Feb 1, 2022 in Education by JackTerrance
0 votes
    I am new to TensorFlow. While I am reading the existing documentation, I found the term tensor really confusing. ... types of tensors? Select the correct answer from above options...
asked Jan 29, 2022 in Education by JackTerrance
0 votes
    What is the difference between the two? It seems that both create new columns, in which their number is equal to ... they are in. Select the correct answer from above options...
asked Feb 1, 2022 in Education by JackTerrance
0 votes
    I am receiving the error: ValueError: Wrong number of items passed 3, placement implies 1, and I am struggling to ... 'sigma'] = sigma Select the correct answer from above options...
asked Feb 1, 2022 in Education by JackTerrance
...