Transformer with Tree-order Encoding for Neural Program Generation

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearch

Authors

While a considerable amount of semantic parsing approaches have employed RNN architectures for code generation tasks, there have been only few attempts to investigate the applicability of Transformers for this task. Including hierarchical information of the underlying programming language syntax has proven to be effective for code generation. Since the positional encoding of the Transformer can only represent positions in a flat sequence, we have extended the encoding scheme to allow the attention mechanism to also attend over hierarchical positions in the input. Furthermore, we have realized a decoder based on a restrictive grammar graph model to improve the generation accuracy and ensure the well-formedness of the generated code. While we did not surpass the state of the art, our findings suggest that employing a tree-based positional encoding in combination with a shared natural-language subword vocabulary improves generation performance over sequential positional encodings.
Original languageEnglish
Title of host publicationConference XXX
Number of pages10
DOIs
Publication statusIn preparation - 30.05.2022
Externally publishedYes

Bibliographical note

This paper was authored in late 2020 and early 2021 for the most part

    Research areas

  • cs.CL, cs.AI, 68T07, 68T50, I.2.7
  • Informatics

Recently viewed

Publications

  1. Der Strommarkt in Europa zwischen Liberalisierung und Klimaschutz
  2. Die Öffentlichkeitsbeteiligung nach der EG-Wasserrahmenrichtlinie
  3. Gefahren und Gefährdungen aus Sicht des Kinder- und Jugendschutzes
  4. Investigation and prediction of grain texture evolution in AA6082
  5. Eine bildungswissenschaftliche Kritik am unternehmerischen Habitus
  6. Patients' and Physicians' Perceptions of Medical Services in Germany
  7. IBaLL - Inklusive Basiskompetenzen in den Fachdidaktiken entwickeln
  8. Measuring and Managing Sustainability Performance of Supply Chains
  9. Die Eigenständigkeit der Bundeswehrverwaltung als Verfassungsgebot
  10. Understanding Sustainability Performance in Business Organizations
  11. Offener Brief zur Ersatzfreiheitsstrafe an die Justizminister*innen
  12. Predicting online user behavior based on Real-Time Advertising Data
  13. Tri‐trophic interaction networks along a tree diversity gradient of BEF‐China
  14. Stakeholders’ Perspectives of Species Diversity in Tree Plantations
  15. Integrating Ecosystem Services in Nature Conservation for Colombia
  16. Spillover of trap-nesting bees and wasps in an urban-rural interface
  17. The Heterogeneous Economic Consequences of Works Council Relations
  18. Natürliche Waldentwicklung in norddeutschen Laubwaldgesellschaften
  19. Auswirkungen des BilRUG auf den handelsrechtlichen Jahresabschluss
  20. Mapping social values of ecosystem services: What is behind the map?
  21. Forest-specific diversity of vascular plants, bryophytes, and lichens
  22. Friction model selection in FEM simulations of aluminium extrusion
  23. Welche Zeitschriften und Journals nehmen Bezug auf die Diagnostica?
  24. Education for Sustainable Consumption through Mindfulness Training
  25. Tage des Mädchenfußballs - eine jugendpädagogische Herausforderung
  26. Centrist Anti-Establishment Parties and Their Struggle for Survival
  27. Nichts als das ? Christoph Brumme erzählt eine autoritäre Erziehung