[FIXED] how to get more than 5000 entities by using Factories GetByFilter method with prestasharp

Issue

Library Version:

1.2.9

NuGet Package Url:

https://www.nuget.org/packages/PrestaSharp/1.2.9

Prestashop version:

1.7.7.0

Describe the Bug:

PrestaSharp GetByFilter with pagination always return same entity list

Since ProductFactory's GetByFilter method returns null if there are more than 5000 products that match the filter. I decide to get them by pagination like this

_productFactory.GetIdsByFilter(filter, null, "[" + startingIndex.ToString() + "," + count.ToString() + "]");

but even if startingIndex(because of a loop) changes the result is the same

Full code:

filter.Add("date_upd", "[" + dFrom + "," + dTo + "]");

int i = 0;

List<long> AllProducts = new List<long>();
List<long> products;
while (true) // this loop never breaks
{ 
    int startingIndex = i++  * count;
    products = _productFactory.GetIdsByFilter(filter, null, "[" + startingIndex.ToString() + "," + (count).ToString() + "]"); // returns same products in every iteration
    if (products?.Any() == true) // to check the list is not empty
    {
        AllProducts.AddRange(products);
        if (products.Count < count)
        {
            break;
        }
    }
    else
        break;
}

Solution

You just have to remove the brackets from the 'limit' parameter. It is an error in the Github documentation when they give the example with brackets. Here's an own implementation where I send multiple requests in parallel to speed up the processing, regards.

public async Task<List<PS_Entity>> GetElements(int pageSize = 100) {
            try {
                var numberOfElements = factory.GetIds().Count;
                var numberOfParallelTasks = numberOfElements >= pageSize ? numberOfElements / pageSize : 1;
                var loadElementsTasks = Enumerable.Range(0, numberOfParallelTasks).Select(taskNumber => factory.GetByFilterAsync(null, "id_ASC", $"{taskNumber * pageSize},{pageSize}")).ToList();
                if (numberOfElements % pageSize != 0) {
                    var skiped = numberOfParallelTasks * pageSize;
                    loadElementsTasks.Add(factory.GetByFilterAsync(null, "id_ASC", $"{skiped},{numberOfElements - skiped}"));
                }
                var elements = (await Task.WhenAll(loadElementsTasks)).SelectMany(elements => elements).ToList();
                return elements;
            } catch (PrestaSharpException e) {
                if ((int)e.ResponseHttpStatusCode == 404)
                    Console.WriteLine($"No existen {RemoveFactoryFromName(factory)}'s.");
                return new List<PS_Entity>();
            }
        }

Answered By - Cristian Rojas

Answer Checked By - Robin (PHPFixing Admin)