[FIXED] Why am I not seeing the headers added by nodejs `response.addTrailers`

Issue

EDIT

I've been able to use curl to contact the backend directly and it is definitely sending the trailing header. It seems to be in the output stream after the body, so it maybe my api routine need to check it later, or it might not be getting past nginx.

2ND EDIT I use curl to access the backend directly I see the trailing header. Contacting the frontend (Nginx) I don't see it

ORIGINAL QUESTION

I am trying to make a node server to respond to API requests from the client.

This server is a backend proxied by nginx, which might be stripping the headers but ... This is its reverse proxy configuration, so I don't think so

  location /api/ {
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_set_header Host $http_host;
    proxy_set_header X-NginX-Proxy true;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_pass http://localhost:2040;
    proxy_redirect off;
    proxy_buffering off;
    proxy_cache off;
  }

The server is processing a particular api request like this

router.post('/api/process', async (req,res) => {
    res.writeHead(200, {
      'Content-Type': 'application/json',
      'Cache-Control': 'no-cache',
      'X-Accel-Buffering': 'no',
      'Trailer': 'API-Status'
    });
    try {
        response = await callApiProcessor(req.params);
        res.write(JSON.stringify(response));
        res.addTrailers({'API-Status': 'OK'})
    catch (e) {
        res.addTrailers({'API-Status': e.toString()});
    }
    res.end();
});

at the browser end, I am using the fetch api like this

export default function api(url, params, signal) {
  const options = {
    credentials: 'same-origin',
    method: 'post',
    headers: new Headers({
      'content-type': 'application/json'
    }),
    body: JSON.stringify(params)
  };
  if (signal) options.signal = signal;
  return window.fetch('/api/' + url, options).then(response => {
    if (!response.ok || response.headers['API-Status'] !== 'OK') {
      if (response.ok) console.warn('SERVER API Error:', response.headers['API-Status']);
      window.dispatchEvent(new LogoffRequest());
      //put us back to home
      window.history.pushState({}, null, '/');
      window.dispatchEvent(new LocationAltered());
      return {};
    } else {
      return response.json();
    }
  });
}

By putting breakpoints in the api module, I cant see any headers in the response. So although response.ok is true, I am still forcing a client log off in my application.

Looking at Chrome's Dev Tools Networking Tab, I cant see the API-Status, although I do see the Trailer header.

Solution

As I noted above, nginx is the strips the trailer. I have asked this question on the nginx mailing list and the answer is that it is not implemented to pass on trailing headers. However, I have solved the problem, which was how can you in the server, notice a problem after you have sent the header (because you've started sending the body) and pass back what is essentially either a 403 or 500 server response.

In the server I have two similar routines - this is the 500 error one, but the 403 is identical except for the status code and I log the ip address of the client (req.headers['x-forwarded-for']).

  function errored(req,res,message) {
    debug('In "Errored"');
    logger('error', `${message} with request url of ${req.originalUrl}`);
    res.statusCode = 500;
    res.end('---500---'); //definitely not json, so should cause api to throw even if attempt to send status code is to no avail.

  }

If this gets called before the body has caused a 200 to get sent, then an actual 500 gets send, else what should in essence be json is interrupted by the text.

Then in the client I have my api calling function:

import { ApiError } from "./events.js";

export default async function api(url, params, signal) {
  const options = {
    credentials: 'same-origin',
    method: 'post',
    headers: new Headers({
      'content-type': 'application/json'
    }),
    body: JSON.stringify(params)
  };
  if (signal) options.signal = signal;
  let text;
  try {
    const response = await window.fetch('/api/' + url, options);
    if (!response.ok) throw new ApiError(response.status); 
    text = await response.text();
    return JSON.parse(text);
  } catch (err) {
    if (err.type === 'api-error') throw err; //just 
      //we failed to parse the json - the actual code should be in the text near the end;
    throw new ApiError(parseInt(text.substr(-6,3),10));    

  }


}

This attempts the request, parses it as text first (you can only read the body once, so we do it as text so we have a copy and then attempt to parse it into JSON). If the final parsing throws, or the response picked up the response error, we can determine which and throw my ApiError function. Elsewhere I have attached an event listener to the uncaught promise rejection

  _promiseRejection(e) {
    if (this.anError) return;
    e.preventDefault();
    const possibleError = e.reason;
    let message
    if (possibleError.type === 'api-error') {
      this._serverError(possibleError)
    } else {
      const message = `Client Error: Uncaught Promise Rejection with reason ${e.reason} has occured`;
      api('/session/log', { type: 'Error', message: message });
      this.dispatchEvent(new SessionStatus({ type: 'error' }));
      this.anError = true;
    }
  }

This last function is in a custom element sitting at the top of my app. Most of the time its contents are hidden, but when the variable this.anError is true it displays the contents . I also have special events to log off users in the case of a 403 (but not a 500) and to switch to the home page if any error.

Answered By - akc42

Answer Checked By - Pedro (PHPFixing Volunteer)