Transformer with Tree-order Encoding for Neural Program Generation

Research output: Contributions to collected editions/worksArticle in conference proceedingsResearch

Authors

While a considerable amount of semantic parsing approaches have employed RNN architectures for code generation tasks, there have been only few attempts to investigate the applicability of Transformers for this task. Including hierarchical information of the underlying programming language syntax has proven to be effective for code generation. Since the positional encoding of the Transformer can only represent positions in a flat sequence, we have extended the encoding scheme to allow the attention mechanism to also attend over hierarchical positions in the input. Furthermore, we have realized a decoder based on a restrictive grammar graph model to improve the generation accuracy and ensure the well-formedness of the generated code. While we did not surpass the state of the art, our findings suggest that employing a tree-based positional encoding in combination with a shared natural-language subword vocabulary improves generation performance over sequential positional encodings.
Original languageEnglish
Title of host publicationConference XXX
Number of pages10
DOIs
Publication statusIn preparation - 30.05.2022
Externally publishedYes

Bibliographical note

This paper was authored in late 2020 and early 2021 for the most part

    Research areas

  • cs.CL, cs.AI, 68T07, 68T50, I.2.7
  • Informatics

Recently viewed

Researchers

  1. Jens Schmidt

Publications

  1. An analytical approach to evaluating nonmonotonic functions of fuzzy numbers
  2. Quantifying diffuse and point inputs of perfluoroalkyl acids in a nonindustrial river catchment
  3. Geometric analysis of a laser scanner functioning based on dynamic triangulation
  4. Second-order SMC with disturbance compensation for robust tracking control in PMSM applications
  5. Late developers and the inequity of "equitable utilization" and the harm of "do no harm"
  6. Enhanced Calculation Procedures for Material and Energy Flow Oriented EMIS
  7. Computing Consumer Sentiment in Germany via Social Media Data
  8. Optimal scheduling for Automated Guided Vehicles (AGV) in blocking job-shops
  9. Do Linguistic Features Influence Item Difficulty in Physics Assessments?
  10. Stabilisierung durch Implementieren, Sichern und Evaluieren
  11. Activating an Integrative Mindset Improves the Subjective Outcomes of Value-Driven Conflicts
  12. Spatial Tests, Familiarity with the Surroundings, and Spatial Activity Experience
  13. Synthesis of Room-Temperature Ionic Liquids with the Weakly Coordinating [Al(ORF)(4)](-) Anion (R-F = C(H)(CF3)(2)) and the Determination of Their Principal Physical Properties
  14. A framework for evaluating the contribution of transformation products to chemical persistence in the environment
  15. “From Within Fur and Feathers”
  16. Software and Web-Based Tools for Sustainability Management in Micro-, Small- and Medium-Sized Enterprises
  17. ”The Machine Could Swallow Everything”
  18. Student Feedback as a Source for Reflection in Practical Phases of Teacher Education
  19. Facets of control
  20. Release of monomers from four different composite materials after halogen and LED curing
  21. Precariousness as a conceptual basis for the understanding of art as uninterrupted primacy of play
  22. Do Exporters Really Pay Higher Wages? First Evidence from German Linked Employer-Employee Data
  23. The importance of school leaders in school health promotion. A European call for systematic integration of health in professional development
  24. Demographic change in work organizations
  25. The Career Engagement Scale: Development and validation of a measure of proactive career behaviors
  26. User experience predicts the effectiveness of a gamified recovery app
  27. Biodegradable green composites
  28. Chasing the difference
  29. Der „reflective practicioner“
  30. Glaspraxis und Umwelttheorie
  31. Elternarbeit in der Krippe
  32. Cruelty and Total War
  33. Re-storying the Business, Innovation and Entrepreneurial Ecosystem Concepts
  34. What are mycorrhizal traits?
  35. Planar Multipol-Resonance-Probe: A Spectral Kinetic Approach
  36. Musikbegriff, erweiterter
  37. Analysis of nutrition-relevant trace elements in human blood and serum by means of total reflection X-ray fluorescence (TXRF) spectroscopy
  38. Atkinson, Anthony B. Inequality. What Can Be Done? Cambridge/Massachusetts. Harvard University Press 2015