Transformer to remove property with value=null from an object

[jon.chleboun]

I have a query that returns a somewhat complex object, and one of the properties consistently has the value of null and that is causing problems for other transforms I want to run - see this property in the sample data below:

"logprobs": null

Sample object:

{
  "id": "cmpl-7vpQOOwncb5RvoaGL59YHIKum9ZlK",
  "object": "text_completion",
  "created": 1694016728,
  "model": "text-davinci-003",
  "choices": [
    {
      "text": "this is a test, this is only a test",
      "index": 0,
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 29,
    "total_tokens": 44
  }
}

How can I remove that property with a Transformer?

Thanks in advance!

[edurmush]

Hello @jon.chleboun,

You can try lodash's omit method.

const data = {
  "id": "cmpl-7vpQOOwncb5RvoaGL59YHIKum9ZlK",
  "object": "text_completion",
  "created": 1694016728,
  "model": "text-davinci-003",
  "choices": [
    {
      "text": "this is a test, this is only a test",
      "index": 0,
      "logprobs": null,
      "finish_reason": "stop"
    }
  ],
  "usage": {
    "prompt_tokens": 15,
    "completion_tokens": 29,
    "total_tokens": 44
  }
};

data.choices = _.map(data.choices, (c) => _.omit(c, 'logprobs'));
return data;

Result:

[jon.chleboun]

@edurmush Thank you so much! That makes a lot of sense and works... almost...

I am seeing this weird thing where when I save and run, the first run it throws an error:

Then it finishes the query, but does not remove the logprobs line:

But then if I run it again, it is successful without the error, and it returns the object without the logprobs line, just like I want it to. When using the app itself (not in the editor) it acts similarly, with an error on the first run, and no error (but also other issues) on the second run after page load.

Any ideas on what could be causing it to complain about the null object on the first run, but then remove it successfully on the second run?

[edurmush]

This is because you are not using the incoming data directly. Redefining the data variable messes things up. Try it without let data = {{....data}} line. Because the data variable will be there in the scope automatically. I put the variable there for demo purposes only.

Like this...

if(data) {
  data.choices = _.map(data.choices, (c) => _.omit(c, 'logprobs'));
}

return data;

[jon.chleboun]

@edurmush Perfect. This did it beautifully. Thank you for the help, I really appreciate it!