Transformer with Tree-order Encoding for Neural Program Generation

Publikation: Beiträge in SammelwerkenAufsätze in KonferenzbändenForschung

Authors

While a considerable amount of semantic parsing approaches have employed RNN architectures for code generation tasks, there have been only few attempts to investigate the applicability of Transformers for this task. Including hierarchical information of the underlying programming language syntax has proven to be effective for code generation. Since the positional encoding of the Transformer can only represent positions in a flat sequence, we have extended the encoding scheme to allow the attention mechanism to also attend over hierarchical positions in the input. Furthermore, we have realized a decoder based on a restrictive grammar graph model to improve the generation accuracy and ensure the well-formedness of the generated code. While we did not surpass the state of the art, our findings suggest that employing a tree-based positional encoding in combination with a shared natural-language subword vocabulary improves generation performance over sequential positional encodings.
OriginalspracheEnglisch
TitelConference XXX
Anzahl der Seiten10
DOIs
PublikationsstatusIn Vorbereitung - 30.05.2022
Extern publiziertJa

Zuletzt angesehen

Publikationen

  1. A Switching Cascade Sliding PID-PID Controllers Combined with a Feedforward and an MPC for an Actuator in Camless Internal Combustion Engines
  2. A Wavelet Packet Algorithm for Online Detection of Pantograph Vibrations
  3. Reality-Based Tasks with Complex-Situations
  4. Environmental Shareholder Value Matrix
  5. The impact of enactive exploration on intrinsic motivation, strategy, and performance in electronic search
  6. A world of abundance
  7. Effects of welding conditions on microstructural transformations and mechanical properties in AE42-HP friction welded joints
  8. Von Modell zu Modell
  9. Battery as a mediating technology of organization
  10. Robust Adaptive Soft Landing Control of an Electromagnetic Valve Actuator for Camless Engines
  11. Origins and practices of genetic risk and responsibility
  12. Over here and over there
  13. Whose Change is it, Anyway?
  14. Where are we with? A dialectical theory on innovation
  15. Cognitive aspects of noise sensitivity
  16. The Contribution of Large Banking Institutions to Systemic Risk
  17. Cultural differences in planning-success relationships
  18. Frontiers of Democracy (Special Issue)
  19. Carabids.org – a dynamic online database of ground beetle species traits (Coleoptera, Carabidae)
  20. Dynamische Losgrößenoptimierung mit bestärkendem Lernen
  21. Kommentierung Art. 13 AEUV
  22. OSZE
  23. Measuring Variation in Gaze Following Across Communities, Ages, and Individuals
  24. Anders als die anderen?
  25. Migration
  26. Internet der Dinge
  27. Migration Struggles and the Global Justice Movement
  28. Action tendencies and characteristics of environmental risks
  29. Putting educational knowledge of prospective teachers to the test
  30. Exploring the Poincaré Ellipsis
  31. Wie wächst das Bildungssystem?
  32. Utilities’ Business Models for Renewable Energy
  33. Global decoupling of functional and phylogenetic diversity in plant communities