The text consists of two strophes with four lines. But in contrast with the first two poems, and consistent with the third poem, the lines rhyme two by two. The poem tells that looking at his love makes him happy, but hearing that she loves him, makes him weep. Is it weeping of happiness? In the context of poem it doesn't seem. In my opinion it seems not very probable that she will say this, so when he thinks about it, he becomes very sad.
Song 4 uses the final D chord of song 3 to start in G. With a very clear and open G chord the song starts with something of the optimism of song 3. But immediately the repeated b by the singer makes it already a little bit melancholic. And starting with the end of m. 2 it goes through e towards a. Starting at the end of m. 2 interpreted by a, we have: V-I-IV-I. But at the end of m. 5 it takes the major parallel C and makes again a V of it through the seventh and goes towards F in measure 6. At the end of m. 7 it transforms by making the II a V7 of C and then goes to the parallel a, stated clearly with II-V-I-II7-V7-and then not a I, but a diminished chord in m. 13. In m. 14-16 it goes from e through the fifth circle towards G, with a suggestion of continuing to C by the seventh in the m. 16, but finding its sad rest in G at the end of the song. So, in contrast with song 1 it starts with a clear harmony and ends with it.
The essence of the poem, when the poet's love says "I love you" and the essence comes that he must cry bitterly, a lot happens in the structure of the music. In m. 12 for the first time the piano and voice are parallel in the repeated tones. Second, immediately afterwards we see a half tone increase in the melody and piano, from G to Gis, creating a diminished chord that gets a lot of attention. Then the motive around "liebe" and "bitterlich" creates a relationship between both. And, finally, the piano repeats a variation on this motive twice, giving it an accent, and also bringing it both to the tonic and to a lower pitch.