PHPFixing
  • Privacy Policy
  • TOS
  • Ask Question
  • Contact Us
  • Home
  • PHP
  • Programming
  • SQL Injection
  • Web3.0

Thursday, March 24, 2022

[FIXED] Codeigniter loop to chunk query only ever gets the first chunk of rows?

 March 24, 2022     codeigniter, mysql, php     No comments   

Issue

I'm trying to get a large amount of data by grabbing it in chunks from the database and writing it to CSV. For some reason the code below is only writing the first chunk (2000 rows) to CSV. I have my $chunk and $limit variables writing to a text file, and those are going through the loop and writing out the correct values. So why isn't $result=$this->db->get('tblProgram', $chunk, $offset)->result_array(); grabbing the next chunks?

Can you not run $this->db->get('tblProgram', $chunk, $offset)->result_array(); multiple times with different offsets? How else would I loop through the results?

I can confirm that I have more than 200k rows returned from the query, and that if I set chunk to something different, I'm still only getting the first chunk returned.

//Get rows from tblTrees based on criteria set in options array in downloadable format
    public function get_download_tree_data($options=array(), $rand=""){

        $this->db->reset_query();
        $this->db->join('tblPlots','tblPlots.programID=tblProgram.pkProgramID');
        $this->db->join('tblTrees','tblTrees.treePlotID=tblPlots.id');
        $this->db->order_by('tblTrees.id', 'ASC');

       // $allResults=$this->db->count_all_results('tblProgram', false);
        $allResults=200000;
        $offset=0;
        $chunk=2000;
        $treePath=$this->config->item('temp_path')."$rand/trees.csv";
        $tree_handle=fopen($treePath,'a');
        $tempPath=$this->config->item('temp_path')."$rand/trees.txt";
        $temp_handle=fopen($tempPath,'a');
        
        while (($offset<$allResults)) {
            $temptxt=$chunk." ".$offset."\n";
            fwrite($temp_handle,$temptxt);

            $result=$this->db->get('tblProgram', $chunk, $offset)->result_array();
            foreach ($result as $row) {
                fputcsv($tree_handle, $row);
            }    
            $offset=$offset+$chunk;
        }
                    
            fclose($tree_handle);
            fclose($temp_handle);
            return array('resultCount'=>$allResults);
 
    }

Solution

https://github.com/bcit-ci/CodeIgniter/blob/develop/system/database/DB_query_builder.php

Looks like calling the get method resets your model:

    public function get($table = '', $limit = NULL, $offset = NULL)
    {
        if ($table !== '')
        {
            $this->_track_aliases($table);
            $this->from($table);
        }

        if ( ! empty($limit))
        {
            $this->limit($limit, $offset);
        }

        $result = $this->query($this->_compile_select());
        $this->_reset_select();
        return $result;
    }

I'd imagine this is the case of any version of ci.



Answered By - Kisaragi
Answer Checked By - Marie Seifert (PHPFixing Admin)
  • Share This:  
  •  Facebook
  •  Twitter
  •  Stumble
  •  Digg
Newer Post Older Post Home
View mobile version

0 Comments:

Post a Comment

Note: Only a member of this blog may post a comment.

Total Pageviews

Featured Post

Why Learn PHP Programming

Why Learn PHP Programming A widely-used open source scripting language PHP is one of the most popular programming languages in the world. It...

Subscribe To

Posts
Atom
Posts
Comments
Atom
Comments

Copyright © PHPFixing