Groupby, apply custom function to data, return results in new columnsCreating a tuple from a CSV filePython Pandas Apply with a Lambda FunctionFaster function iteration on Pandas df with axis=1Grouping and aggregating using PandasCalculating sum of multiple columns in pandasConvert seconds to hours/minutes/seconds and pretty printCreate new columns using groupby in pandasPandas: missing value imputationprocess data with numpy seems rather slow than pure python?Calculating Relative Strength Index for Multiple Pandas column

Why is stat::st_size 0 for devices but at the same time lseek defines the device size correctly?

What is IP squat space

RegionDifference for Cylinder and Cuboid

Identifying the interval from A♭ to D♯

How to make healing in an exploration game interesting

How is the Swiss post e-voting system supposed to work, and how was it wrong?

Theorems like the Lovász Local Lemma?

Can hydraulic brake levers get hot when brakes overheat?

My story is written in English, but is set in my home country. What language should I use for the dialogue?

At what level can a dragon innately cast its spells?

The use of "touch" and "touch on" in context

Do I need life insurance if I can cover my own funeral costs?

How could a scammer know the apps on my phone / iTunes account?

Why are the outputs of printf and std::cout different

Good allowance savings plan?

Using "wallow" verb with object

Humanity loses the vast majority of its technology, information, and population in the year 2122. How long does it take to rebuild itself?

Employee lack of ownership

Making a sword in the stone, in a medieval world without magic

Official degrees of earth’s rotation per day

Why does Deadpool say "You're welcome, Canada," after shooting Ryan Reynolds in the end credits?

Should we release the security issues we found in our product as CVE or we can just update those on weekly release notes?

Connecting top and bottom SMD component pads using via

Fill color and outline color with the same value



Groupby, apply custom function to data, return results in new columns


Creating a tuple from a CSV filePython Pandas Apply with a Lambda FunctionFaster function iteration on Pandas df with axis=1Grouping and aggregating using PandasCalculating sum of multiple columns in pandasConvert seconds to hours/minutes/seconds and pretty printCreate new columns using groupby in pandasPandas: missing value imputationprocess data with numpy seems rather slow than pure python?Calculating Relative Strength Index for Multiple Pandas column













0












$begingroup$


I want to group by id, apply a custom function to the data, and create a new column with the results. It seems there must be a faster/more efficient way to do this than to pass the data to the function, make the changes, and return the data. Here is an example.



Example



dat = pd.DataFrame('id': ['a', 'a', 'a', 'b', 'b', 'b'], 'x': [4, 8, 12, 25, 30, 50])

def my_func(data):
data['diff'] = (data['x'] - data['x'].shift(1, fill_value=data['x'].iat[0]))
return data

dat.groupby('id').apply(my_func)


Output



> print(dat)


id x diff
0 a 4 0
1 a 8 4
2 a 12 4
3 b 25 0
4 b 30 5
5 b 50 20


Is there a more efficient way to do this?









share









$endgroup$
















    0












    $begingroup$


    I want to group by id, apply a custom function to the data, and create a new column with the results. It seems there must be a faster/more efficient way to do this than to pass the data to the function, make the changes, and return the data. Here is an example.



    Example



    dat = pd.DataFrame('id': ['a', 'a', 'a', 'b', 'b', 'b'], 'x': [4, 8, 12, 25, 30, 50])

    def my_func(data):
    data['diff'] = (data['x'] - data['x'].shift(1, fill_value=data['x'].iat[0]))
    return data

    dat.groupby('id').apply(my_func)


    Output



    > print(dat)


    id x diff
    0 a 4 0
    1 a 8 4
    2 a 12 4
    3 b 25 0
    4 b 30 5
    5 b 50 20


    Is there a more efficient way to do this?









    share









    $endgroup$














      0












      0








      0





      $begingroup$


      I want to group by id, apply a custom function to the data, and create a new column with the results. It seems there must be a faster/more efficient way to do this than to pass the data to the function, make the changes, and return the data. Here is an example.



      Example



      dat = pd.DataFrame('id': ['a', 'a', 'a', 'b', 'b', 'b'], 'x': [4, 8, 12, 25, 30, 50])

      def my_func(data):
      data['diff'] = (data['x'] - data['x'].shift(1, fill_value=data['x'].iat[0]))
      return data

      dat.groupby('id').apply(my_func)


      Output



      > print(dat)


      id x diff
      0 a 4 0
      1 a 8 4
      2 a 12 4
      3 b 25 0
      4 b 30 5
      5 b 50 20


      Is there a more efficient way to do this?









      share









      $endgroup$




      I want to group by id, apply a custom function to the data, and create a new column with the results. It seems there must be a faster/more efficient way to do this than to pass the data to the function, make the changes, and return the data. Here is an example.



      Example



      dat = pd.DataFrame('id': ['a', 'a', 'a', 'b', 'b', 'b'], 'x': [4, 8, 12, 25, 30, 50])

      def my_func(data):
      data['diff'] = (data['x'] - data['x'].shift(1, fill_value=data['x'].iat[0]))
      return data

      dat.groupby('id').apply(my_func)


      Output



      > print(dat)


      id x diff
      0 a 4 0
      1 a 8 4
      2 a 12 4
      3 b 25 0
      4 b 30 5
      5 b 50 20


      Is there a more efficient way to do this?







      python pandas





      share












      share










      share



      share










      asked 1 min ago









      AmstellAmstell

      7611




      7611




















          0






          active

          oldest

          votes











          Your Answer





          StackExchange.ifUsing("editor", function ()
          return StackExchange.using("mathjaxEditing", function ()
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
          );
          );
          , "mathjax-editing");

          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "196"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          autoActivateHeartbeat: false,
          convertImagesToLinks: false,
          noModals: true,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          imageUploader:
          brandingHtml: "Powered by u003ca class="icon-imgur-white" href="https://imgur.com/"u003eu003c/au003e",
          contentPolicyHtml: "User contributions licensed under u003ca href="https://creativecommons.org/licenses/by-sa/3.0/"u003ecc by-sa 3.0 with attribution requiredu003c/au003e u003ca href="https://stackoverflow.com/legal/content-policy"u003e(content policy)u003c/au003e",
          allowUrls: true
          ,
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );













          draft saved

          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f215465%2fgroupby-apply-custom-function-to-data-return-results-in-new-columns%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown

























          0






          active

          oldest

          votes








          0






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes















          draft saved

          draft discarded
















































          Thanks for contributing an answer to Code Review Stack Exchange!


          • Please be sure to answer the question. Provide details and share your research!

          But avoid


          • Asking for help, clarification, or responding to other answers.

          • Making statements based on opinion; back them up with references or personal experience.

          Use MathJax to format equations. MathJax reference.


          To learn more, see our tips on writing great answers.




          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f215465%2fgroupby-apply-custom-function-to-data-return-results-in-new-columns%23new-answer', 'question_page');

          );

          Post as a guest















          Required, but never shown





















































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown

































          Required, but never shown














          Required, but never shown












          Required, but never shown







          Required, but never shown







          Popular posts from this blog

          कुँवर स्रोत दिक्चालन सूची"कुँवर""राणा कुँवरके वंशावली"

          Why is a white electrical wire connected to 2 black wires?How to wire a light fixture with 3 white wires in box?How should I wire a ceiling fan when there's only three wires in the box?Two white, two black, two ground, and red wire in ceiling box connected to switchWhy is there a white wire connected to multiple black wires in my light box?How to wire a light with two white wires and one black wireReplace light switch connected to a power outlet with dimmer - two black wires to one black and redHow to wire a light with multiple black/white/green wires from the ceiling?Ceiling box has 2 black and white wires but fan/ light only has 1 of eachWhy neutral wire connected to load wire?Switch with 2 black, 2 white, 2 ground and 1 red wire connected to ceiling light and a receptacle?

          चैत्य भूमि चित्र दीर्घा सन्दर्भ बाहरी कडियाँ दिक्चालन सूची"Chaitya Bhoomi""Chaitya Bhoomi: Statue of Equality in India""Dadar Chaitya Bhoomi: Statue of Equality in India""Ambedkar memorial: Centre okays transfer of Indu Mill land"चैत्यभमि