== Physical Plan ==
AdaptiveSparkPlan (849)
+- == Final Plan ==
ResultQueryStage (844), Statistics(sizeInBytes=8.0 EiB)
+- Execute InsertIntoHadoopFsRelationCommand (843)
+- WriteFiles (842)
+- Project (841)
+- Filter (840)
+- TableCacheQueryStage (839), Statistics(sizeInBytes=6.30E+161 B)
+- InMemoryTableScan (1)
+- InMemoryRelation (2)
+- AdaptiveSparkPlan (838)
+- == Final Plan ==
ResultQueryStage (828)
+- * Filter (827)
+- ObjectHashAggregate (826)
+- ShuffleQueryStage (825), Statistics(sizeInBytes=13.0 KiB, rowCount=41)
+- Exchange (824)
+- ObjectHashAggregate (823)
+- Project (822)
+- Window (821)
+- * Sort (820)
+- AQEShuffleRead (819)
+- ShuffleQueryStage (818), Statistics(sizeInBytes=17.7 KiB, rowCount=41)
+- Exchange (817)
+- * Project (816)
+- TableCacheQueryStage (815), Statistics(sizeInBytes=2.73E+162 B)
+- InMemoryTableScan (3)
+- InMemoryRelation (4)
+- AdaptiveSparkPlan (814)
+- == Final Plan ==
ResultQueryStage (321)
+- Project (320)
+- * BroadcastHashJoin Inner BuildRight (319)
:- * Project (307)
: +- * BroadcastHashJoin Inner BuildLeft (306)
: :- BroadcastQueryStage (290), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (289)
: : +- AQEShuffleRead (288)
: : +- ShuffleQueryStage (287), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : +- Exchange (286)
: : +- * Project (285)
: : +- * BroadcastHashJoin LeftOuter BuildRight (284)
: : :- AQEShuffleRead (178)
: : : +- ShuffleQueryStage (177), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : : +- Exchange (176)
: : : +- * Project (175)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (174)
: : : :- * Project (42)
: : : : +- * Generate (41)
: : : : +- * Project (40)
: : : : +- * Generate (39)
: : : : +- Project (38)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (37)
: : : : :- * Project (28)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (27)
: : : : : :- * Project (19)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (18)
: : : : : : :- ShuffleQueryStage (9), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (8)
: : : : : : : +- * Filter (7)
: : : : : : : +- * ColumnarToRow (6)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastQueryStage (17), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (16)
: : : : : : +- * Project (15)
: : : : : : +- ShuffleQueryStage (14), Statistics(sizeInBytes=339.5 KiB, rowCount=2.75E+3)
: : : : : : +- Exchange (13)
: : : : : : +- * Project (12)
: : : : : : +- * ColumnarToRow (11)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastQueryStage (26), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- BroadcastExchange (25)
: : : : : +- ShuffleQueryStage (24), Statistics(sizeInBytes=4.4 KiB, rowCount=62)
: : : : : +- Exchange (23)
: : : : : +- * Project (22)
: : : : : +- * ColumnarToRow (21)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastQueryStage (36), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- BroadcastExchange (35)
: : : : +- ShuffleQueryStage (34), Statistics(sizeInBytes=92.7 KiB, rowCount=159)
: : : : +- Exchange (33)
: : : : +- * Project (32)
: : : : +- * Filter (31)
: : : : +- * ColumnarToRow (30)
: : : : +- Scan parquet (29)
: : : +- BroadcastQueryStage (173), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (172)
: : : +- * HashAggregate (171)
: : : +- AQEShuffleRead (170)
: : : +- ShuffleQueryStage (169), Statistics(sizeInBytes=27.7 KiB, rowCount=295)
: : : +- Exchange (168)
: : : +- * HashAggregate (167)
: : : +- * Project (166)
: : : +- * BroadcastHashJoin Inner BuildLeft (165)
: : : :- BroadcastQueryStage (137), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (136)
: : : : +- AQEShuffleRead (135)
: : : : +- ShuffleQueryStage (134), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : : +- Exchange (133)
: : : : +- * Project (132)
: : : : +- * BroadcastHashJoin Inner BuildLeft (131)
: : : : :- BroadcastQueryStage (109), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (108)
: : : : : +- AQEShuffleRead (107)
: : : : : +- ShuffleQueryStage (106), Statistics(sizeInBytes=43.1 KiB, rowCount=394)
: : : : : +- Exchange (105)
: : : : : +- * Project (104)
: : : : : +- * Filter (103)
: : : : : +- * Generate (102)
: : : : : +- * Filter (101)
: : : : : +- ObjectHashAggregate (100)
: : : : : +- ObjectHashAggregate (99)
: : : : : +- Generate (98)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (97)
: : : : : :- BroadcastQueryStage (50), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- BroadcastExchange (49)
: : : : : : +- ShuffleQueryStage (48), Statistics(sizeInBytes=240.1 KiB, rowCount=398)
: : : : : : +- Exchange (47)
: : : : : : +- * Project (46)
: : : : : : +- * Filter (45)
: : : : : : +- * ColumnarToRow (44)
: : : : : : +- Scan parquet (43)
: : : : : +- * Project (96)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (95)
: : : : : :- BroadcastQueryStage (63), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- BroadcastExchange (62)
: : : : : : +- * Project (61)
: : : : : : +- * Generate (60)
: : : : : : +- * Project (59)
: : : : : : +- * Filter (58)
: : : : : : +- * Generate (57)
: : : : : : +- ShuffleQueryStage (56), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : : : : : +- Exchange (55)
: : : : : : +- * Project (54)
: : : : : : +- * Filter (53)
: : : : : : +- * ColumnarToRow (52)
: : : : : : +- Scan parquet (51)
: : : : : +- * HashAggregate (94)
: : : : : +- AQEShuffleRead (93)
: : : : : +- ShuffleQueryStage (92), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- Exchange (91)
: : : : : +- * HashAggregate (90)
: : : : : +- * Project (89)
: : : : : +- * Generate (88)
: : : : : +- * Project (87)
: : : : : +- * Generate (86)
: : : : : +- Project (85)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (84)
: : : : : :- * Project (81)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (80)
: : : : : : :- * Project (72)
: : : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (71)
: : : : : : : :- ShuffleQueryStage (68), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : : : : : : : +- Exchange (67)
: : : : : : : : +- * Filter (66)
: : : : : : : : +- * ColumnarToRow (65)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastQueryStage (70), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : : +- ReusedExchange (69)
: : : : : : +- BroadcastQueryStage (79), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : : +- BroadcastExchange (78)
: : : : : : +- ShuffleQueryStage (77), Statistics(sizeInBytes=2.9 KiB, rowCount=62)
: : : : : : +- Exchange (76)
: : : : : : +- * Project (75)
: : : : : : +- * ColumnarToRow (74)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastQueryStage (83), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : : +- ReusedExchange (82)
: : : : +- * Project (130)
: : : : +- * Generate (129)
: : : : +- * Project (128)
: : : : +- * Generate (127)
: : : : +- Project (126)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (125)
: : : : :- * Project (122)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (121)
: : : : : :- * Project (118)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (117)
: : : : : : :- ShuffleQueryStage (114), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (113)
: : : : : : : +- * Filter (112)
: : : : : : : +- * ColumnarToRow (111)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastQueryStage (116), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (115)
: : : : : +- BroadcastQueryStage (120), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (119)
: : : : +- BroadcastQueryStage (124), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (123)
: : : +- AQEShuffleRead (164)
: : : +- ShuffleQueryStage (163), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- Exchange (162)
: : : +- * HashAggregate (161)
: : : +- * HashAggregate (160)
: : : +- * Project (159)
: : : +- * BroadcastHashJoin Inner BuildLeft (158)
: : : :- BroadcastQueryStage (139), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : +- ReusedExchange (138)
: : : +- * Project (157)
: : : +- * Generate (156)
: : : +- * Project (155)
: : : +- * Generate (154)
: : : +- Project (153)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (152)
: : : :- * Project (149)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (148)
: : : : :- * Project (145)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (144)
: : : : : :- ShuffleQueryStage (141), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : +- ReusedExchange (140)
: : : : : +- BroadcastQueryStage (143), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : +- ReusedExchange (142)
: : : : +- BroadcastQueryStage (147), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : +- ReusedExchange (146)
: : : +- BroadcastQueryStage (151), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : +- ReusedExchange (150)
: : +- BroadcastQueryStage (283), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (282)
: : +- AQEShuffleRead (281)
: : +- ShuffleQueryStage (280), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : +- Exchange (279)
: : +- * SerializeFromObject (278)
: : +- * MapElements (277)
: : +- * DeserializeToObject (276)
: : +- ObjectHashAggregate (275)
: : +- ObjectHashAggregate (274)
: : +- * Project (273)
: : +- * BroadcastHashJoin LeftOuter BuildRight (272)
: : :- * Project (266)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (265)
: : : :- * Project (204)
: : : : +- * Project (203)
: : : : +- * Generate (202)
: : : : +- * Generate (201)
: : : : +- * Project (200)
: : : : +- Project (199)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (198)
: : : : :- * Project (195)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (194)
: : : : : :- * BroadcastHashJoin LeftOuter BuildRight (191)
: : : : : : :- ShuffleQueryStage (182), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (181)
: : : : : : : +- * ColumnarToRow (180)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastQueryStage (190), Statistics(sizeInBytes=4.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (189)
: : : : : : +- * Project (188)
: : : : : : +- ShuffleQueryStage (187), Statistics(sizeInBytes=2.0 MiB, rowCount=2.75E+3)
: : : : : : +- Exchange (186)
: : : : : : +- * Project (185)
: : : : : : +- * ColumnarToRow (184)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastQueryStage (193), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (192)
: : : : +- BroadcastQueryStage (197), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (196)
: : : +- BroadcastQueryStage (264), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (263)
: : : +- * HashAggregate (262)
: : : +- AQEShuffleRead (261)
: : : +- ShuffleQueryStage (260), Statistics(sizeInBytes=30.0 KiB, rowCount=295)
: : : +- Exchange (259)
: : : +- * HashAggregate (258)
: : : +- * Project (257)
: : : +- * BroadcastHashJoin Inner BuildLeft (256)
: : : :- BroadcastQueryStage (252), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (251)
: : : : +- AQEShuffleRead (250)
: : : : +- ShuffleQueryStage (249), Statistics(sizeInBytes=34.6 KiB, rowCount=295)
: : : : +- Exchange (248)
: : : : +- * Project (247)
: : : : +- * BroadcastHashJoin Inner BuildLeft (246)
: : : : :- BroadcastQueryStage (227), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (226)
: : : : : +- AQEShuffleRead (225)
: : : : : +- ShuffleQueryStage (224), Statistics(sizeInBytes=46.2 KiB, rowCount=394)
: : : : : +- Exchange (223)
: : : : : +- * Project (222)
: : : : : +- * Filter (221)
: : : : : +- * Generate (220)
: : : : : +- * Filter (219)
: : : : : +- ObjectHashAggregate (218)
: : : : : +- ObjectHashAggregate (217)
: : : : : +- Generate (216)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (215)
: : : : : :- BroadcastQueryStage (206), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- ReusedExchange (205)
: : : : : +- * Project (214)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (213)
: : : : : :- BroadcastQueryStage (208), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- ReusedExchange (207)
: : : : : +- * HashAggregate (212)
: : : : : +- AQEShuffleRead (211)
: : : : : +- ShuffleQueryStage (210), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- ReusedExchange (209)
: : : : +- * Project (245)
: : : : +- * Generate (244)
: : : : +- * Project (243)
: : : : +- * Generate (242)
: : : : +- Project (241)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (240)
: : : : :- * Project (237)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (236)
: : : : : :- * Project (233)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (232)
: : : : : : :- ShuffleQueryStage (229), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- ReusedExchange (228)
: : : : : : +- BroadcastQueryStage (231), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (230)
: : : : : +- BroadcastQueryStage (235), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (234)
: : : : +- BroadcastQueryStage (239), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (238)
: : : +- AQEShuffleRead (255)
: : : +- ShuffleQueryStage (254), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- ReusedExchange (253)
: : +- BroadcastQueryStage (271), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (270)
: : +- * Filter (269)
: : +- * ColumnarToRow (268)
: : +- Scan parquet (267)
: +- * Project (305)
: +- * BroadcastHashJoin Inner BuildLeft (304)
: :- BroadcastQueryStage (300), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : +- BroadcastExchange (299)
: : +- * Project (298)
: : +- * Filter (297)
: : +- * Generate (296)
: : +- * Project (295)
: : +- * Filter (294)
: : +- * Generate (293)
: : +- ShuffleQueryStage (292), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : +- ReusedExchange (291)
: +- * HashAggregate (303)
: +- ShuffleQueryStage (302), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: +- ReusedExchange (301)
+- BroadcastQueryStage (318), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
+- BroadcastExchange (317)
+- * Project (316)
+- * Filter (315)
+- * Generate (314)
+- ShuffleQueryStage (313), Statistics(sizeInBytes=29.9 KiB, rowCount=9)
+- Exchange (312)
+- * Project (311)
+- * Filter (310)
+- * ColumnarToRow (309)
+- Scan parquet (308)
+- == Initial Plan ==
Project (813)
+- BroadcastHashJoin Inner BuildRight (812)
:- Project (804)
: +- SortMergeJoin Inner (803)
: :- Sort (758)
: : +- Exchange (757)
: : +- Project (756)
: : +- SortMergeJoin LeftOuter (755)
: : :- Sort (530)
: : : +- Exchange (529)
: : : +- Project (528)
: : : +- SortMergeJoin LeftOuter (527)
: : : :- Sort (345)
: : : : +- Project (344)
: : : : +- Generate (343)
: : : : +- Project (342)
: : : : +- Generate (341)
: : : : +- Project (340)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (339)
: : : : :- Project (334)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (333)
: : : : : :- Project (329)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (328)
: : : : : : :- Exchange (323)
: : : : : : : +- Filter (322)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastExchange (327)
: : : : : : +- Project (326)
: : : : : : +- Exchange (325)
: : : : : : +- Project (324)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastExchange (332)
: : : : : +- Exchange (331)
: : : : : +- Project (330)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastExchange (338)
: : : : +- Exchange (337)
: : : : +- Project (336)
: : : : +- Filter (335)
: : : : +- Scan parquet (29)
: : : +- Sort (526)
: : : +- HashAggregate (525)
: : : +- Exchange (524)
: : : +- HashAggregate (523)
: : : +- Project (522)
: : : +- SortMergeJoin Inner (521)
: : : :- Sort (429)
: : : : +- Exchange (428)
: : : : +- Project (427)
: : : : +- SortMergeJoin Inner (426)
: : : : :- Sort (398)
: : : : : +- Exchange (397)
: : : : : +- Project (396)
: : : : : +- Filter (395)
: : : : : +- Generate (394)
: : : : : +- Filter (393)
: : : : : +- ObjectHashAggregate (392)
: : : : : +- ObjectHashAggregate (391)
: : : : : +- Generate (390)
: : : : : +- BroadcastHashJoin Inner BuildLeft (389)
: : : : : :- BroadcastExchange (349)
: : : : : : +- Exchange (348)
: : : : : : +- Project (347)
: : : : : : +- Filter (346)
: : : : : : +- Scan parquet (43)
: : : : : +- Project (388)
: : : : : +- BroadcastHashJoin Inner BuildLeft (387)
: : : : : :- BroadcastExchange (358)
: : : : : : +- Project (357)
: : : : : : +- Generate (356)
: : : : : : +- Project (355)
: : : : : : +- Filter (354)
: : : : : : +- Generate (353)
: : : : : : +- Exchange (352)
: : : : : : +- Project (351)
: : : : : : +- Filter (350)
: : : : : : +- Scan parquet (51)
: : : : : +- HashAggregate (386)
: : : : : +- Exchange (385)
: : : : : +- HashAggregate (384)
: : : : : +- Project (383)
: : : : : +- Generate (382)
: : : : : +- Project (381)
: : : : : +- Generate (380)
: : : : : +- Project (379)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (378)
: : : : : :- Project (372)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (371)
: : : : : : :- Project (367)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (366)
: : : : : : : :- Exchange (360)
: : : : : : : : +- Filter (359)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastExchange (365)
: : : : : : : +- Project (364)
: : : : : : : +- Exchange (363)
: : : : : : : +- Project (362)
: : : : : : : +- Scan parquet (361)
: : : : : : +- BroadcastExchange (370)
: : : : : : +- Exchange (369)
: : : : : : +- Project (368)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastExchange (377)
: : : : : +- Exchange (376)
: : : : : +- Project (375)
: : : : : +- Filter (374)
: : : : : +- Scan parquet (373)
: : : : +- Sort (425)
: : : : +- Project (424)
: : : : +- Generate (423)
: : : : +- Project (422)
: : : : +- Generate (421)
: : : : +- Project (420)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (419)
: : : : :- Project (413)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (412)
: : : : : :- Project (407)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (406)
: : : : : : :- Exchange (400)
: : : : : : : +- Filter (399)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastExchange (405)
: : : : : : +- Project (404)
: : : : : : +- Exchange (403)
: : : : : : +- Project (402)
: : : : : : +- Scan parquet (401)
: : : : : +- BroadcastExchange (411)
: : : : : +- Exchange (410)
: : : : : +- Project (409)
: : : : : +- Scan parquet (408)
: : : : +- BroadcastExchange (418)
: : : : +- Exchange (417)
: : : : +- Project (416)
: : : : +- Filter (415)
: : : : +- Scan parquet (414)
: : : +- Sort (520)
: : : +- Exchange (519)
: : : +- HashAggregate (518)
: : : +- HashAggregate (517)
: : : +- Project (516)
: : : +- SortMergeJoin Inner (515)
: : : :- Sort (486)
: : : : +- Exchange (485)
: : : : +- Project (484)
: : : : +- Filter (483)
: : : : +- Generate (482)
: : : : +- Filter (481)
: : : : +- ObjectHashAggregate (480)
: : : : +- ObjectHashAggregate (479)
: : : : +- Generate (478)
: : : : +- BroadcastHashJoin Inner BuildLeft (477)
: : : : :- BroadcastExchange (434)
: : : : : +- Exchange (433)
: : : : : +- Project (432)
: : : : : +- Filter (431)
: : : : : +- Scan parquet (430)
: : : : +- Project (476)
: : : : +- BroadcastHashJoin Inner BuildLeft (475)
: : : : :- BroadcastExchange (444)
: : : : : +- Project (443)
: : : : : +- Generate (442)
: : : : : +- Project (441)
: : : : : +- Filter (440)
: : : : : +- Generate (439)
: : : : : +- Exchange (438)
: : : : : +- Project (437)
: : : : : +- Filter (436)
: : : : : +- Scan parquet (435)
: : : : +- HashAggregate (474)
: : : : +- Exchange (473)
: : : : +- HashAggregate (472)
: : : : +- Project (471)
: : : : +- Generate (470)
: : : : +- Project (469)
: : : : +- Generate (468)
: : : : +- Project (467)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (466)
: : : : :- Project (460)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (459)
: : : : : :- Project (454)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (453)
: : : : : : :- Exchange (447)
: : : : : : : +- Filter (446)
: : : : : : : +- Scan parquet (445)
: : : : : : +- BroadcastExchange (452)
: : : : : : +- Project (451)
: : : : : : +- Exchange (450)
: : : : : : +- Project (449)
: : : : : : +- Scan parquet (448)
: : : : : +- BroadcastExchange (458)
: : : : : +- Exchange (457)
: : : : : +- Project (456)
: : : : : +- Scan parquet (455)
: : : : +- BroadcastExchange (465)
: : : : +- Exchange (464)
: : : : +- Project (463)
: : : : +- Filter (462)
: : : : +- Scan parquet (461)
: : : +- Sort (514)
: : : +- Project (513)
: : : +- Generate (512)
: : : +- Project (511)
: : : +- Generate (510)
: : : +- Project (509)
: : : +- BroadcastHashJoin LeftOuter BuildRight (508)
: : : :- Project (502)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (501)
: : : : :- Project (496)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (495)
: : : : : :- Exchange (489)
: : : : : : +- Filter (488)
: : : : : : +- Scan parquet (487)
: : : : : +- BroadcastExchange (494)
: : : : : +- Project (493)
: : : : : +- Exchange (492)
: : : : : +- Project (491)
: : : : : +- Scan parquet (490)
: : : : +- BroadcastExchange (500)
: : : : +- Exchange (499)
: : : : +- Project (498)
: : : : +- Scan parquet (497)
: : : +- BroadcastExchange (507)
: : : +- Exchange (506)
: : : +- Project (505)
: : : +- Filter (504)
: : : +- Scan parquet (503)
: : +- Sort (754)
: : +- Exchange (753)
: : +- SerializeFromObject (752)
: : +- MapElements (751)
: : +- DeserializeToObject (750)
: : +- ObjectHashAggregate (749)
: : +- ObjectHashAggregate (748)
: : +- Project (747)
: : +- BroadcastHashJoin LeftOuter BuildRight (746)
: : :- Project (743)
: : : +- SortMergeJoin LeftOuter (742)
: : : :- Sort (555)
: : : : +- Project (554)
: : : : +- Project (553)
: : : : +- Generate (552)
: : : : +- Generate (551)
: : : : +- Project (550)
: : : : +- Project (549)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (548)
: : : : :- Project (542)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (541)
: : : : : :- BroadcastHashJoin LeftOuter BuildRight (536)
: : : : : : :- Exchange (531)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastExchange (535)
: : : : : : +- Project (534)
: : : : : : +- Exchange (533)
: : : : : : +- Project (532)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastExchange (540)
: : : : : +- Exchange (539)
: : : : : +- Project (538)
: : : : : +- Scan parquet (537)
: : : : +- BroadcastExchange (547)
: : : : +- Exchange (546)
: : : : +- Project (545)
: : : : +- Filter (544)
: : : : +- Scan parquet (543)
: : : +- Sort (741)
: : : +- HashAggregate (740)
: : : +- Exchange (739)
: : : +- HashAggregate (738)
: : : +- Project (737)
: : : +- SortMergeJoin Inner (736)
: : : :- Sort (644)
: : : : +- Exchange (643)
: : : : +- Project (642)
: : : : +- SortMergeJoin Inner (641)
: : : : :- Sort (612)
: : : : : +- Exchange (611)
: : : : : +- Project (610)
: : : : : +- Filter (609)
: : : : : +- Generate (608)
: : : : : +- Filter (607)
: : : : : +- ObjectHashAggregate (606)
: : : : : +- ObjectHashAggregate (605)
: : : : : +- Generate (604)
: : : : : +- BroadcastHashJoin Inner BuildLeft (603)
: : : : : :- BroadcastExchange (560)
: : : : : : +- Exchange (559)
: : : : : : +- Project (558)
: : : : : : +- Filter (557)
: : : : : : +- Scan parquet (556)
: : : : : +- Project (602)
: : : : : +- BroadcastHashJoin Inner BuildLeft (601)
: : : : : :- BroadcastExchange (570)
: : : : : : +- Project (569)
: : : : : : +- Generate (568)
: : : : : : +- Project (567)
: : : : : : +- Filter (566)
: : : : : : +- Generate (565)
: : : : : : +- Exchange (564)
: : : : : : +- Project (563)
: : : : : : +- Filter (562)
: : : : : : +- Scan parquet (561)
: : : : : +- HashAggregate (600)
: : : : : +- Exchange (599)
: : : : : +- HashAggregate (598)
: : : : : +- Project (597)
: : : : : +- Generate (596)
: : : : : +- Project (595)
: : : : : +- Generate (594)
: : : : : +- Project (593)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (592)
: : : : : :- Project (586)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (585)
: : : : : : :- Project (580)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (579)
: : : : : : : :- Exchange (573)
: : : : : : : : +- Filter (572)
: : : : : : : : +- Scan parquet (571)
: : : : : : : +- BroadcastExchange (578)
: : : : : : : +- Project (577)
: : : : : : : +- Exchange (576)
: : : : : : : +- Project (575)
: : : : : : : +- Scan parquet (574)
: : : : : : +- BroadcastExchange (584)
: : : : : : +- Exchange (583)
: : : : : : +- Project (582)
: : : : : : +- Scan parquet (581)
: : : : : +- BroadcastExchange (591)
: : : : : +- Exchange (590)
: : : : : +- Project (589)
: : : : : +- Filter (588)
: : : : : +- Scan parquet (587)
: : : : +- Sort (640)
: : : : +- Project (639)
: : : : +- Generate (638)
: : : : +- Project (637)
: : : : +- Generate (636)
: : : : +- Project (635)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (634)
: : : : :- Project (628)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (627)
: : : : : :- Project (622)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (621)
: : : : : : :- Exchange (615)
: : : : : : : +- Filter (614)
: : : : : : : +- Scan parquet (613)
: : : : : : +- BroadcastExchange (620)
: : : : : : +- Project (619)
: : : : : : +- Exchange (618)
: : : : : : +- Project (617)
: : : : : : +- Scan parquet (616)
: : : : : +- BroadcastExchange (626)
: : : : : +- Exchange (625)
: : : : : +- Project (624)
: : : : : +- Scan parquet (623)
: : : : +- BroadcastExchange (633)
: : : : +- Exchange (632)
: : : : +- Project (631)
: : : : +- Filter (630)
: : : : +- Scan parquet (629)
: : : +- Sort (735)
: : : +- Exchange (734)
: : : +- HashAggregate (733)
: : : +- HashAggregate (732)
: : : +- Project (731)
: : : +- SortMergeJoin Inner (730)
: : : :- Sort (701)
: : : : +- Exchange (700)
: : : : +- Project (699)
: : : : +- Filter (698)
: : : : +- Generate (697)
: : : : +- Filter (696)
: : : : +- ObjectHashAggregate (695)
: : : : +- ObjectHashAggregate (694)
: : : : +- Generate (693)
: : : : +- BroadcastHashJoin Inner BuildLeft (692)
: : : : :- BroadcastExchange (649)
: : : : : +- Exchange (648)
: : : : : +- Project (647)
: : : : : +- Filter (646)
: : : : : +- Scan parquet (645)
: : : : +- Project (691)
: : : : +- BroadcastHashJoin Inner BuildLeft (690)
: : : : :- BroadcastExchange (659)
: : : : : +- Project (658)
: : : : : +- Generate (657)
: : : : : +- Project (656)
: : : : : +- Filter (655)
: : : : : +- Generate (654)
: : : : : +- Exchange (653)
: : : : : +- Project (652)
: : : : : +- Filter (651)
: : : : : +- Scan parquet (650)
: : : : +- HashAggregate (689)
: : : : +- Exchange (688)
: : : : +- HashAggregate (687)
: : : : +- Project (686)
: : : : +- Generate (685)
: : : : +- Project (684)
: : : : +- Generate (683)
: : : : +- Project (682)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (681)
: : : : :- Project (675)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (674)
: : : : : :- Project (669)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (668)
: : : : : : :- Exchange (662)
: : : : : : : +- Filter (661)
: : : : : : : +- Scan parquet (660)
: : : : : : +- BroadcastExchange (667)
: : : : : : +- Project (666)
: : : : : : +- Exchange (665)
: : : : : : +- Project (664)
: : : : : : +- Scan parquet (663)
: : : : : +- BroadcastExchange (673)
: : : : : +- Exchange (672)
: : : : : +- Project (671)
: : : : : +- Scan parquet (670)
: : : : +- BroadcastExchange (680)
: : : : +- Exchange (679)
: : : : +- Project (678)
: : : : +- Filter (677)
: : : : +- Scan parquet (676)
: : : +- Sort (729)
: : : +- Project (728)
: : : +- Generate (727)
: : : +- Project (726)
: : : +- Generate (725)
: : : +- Project (724)
: : : +- BroadcastHashJoin LeftOuter BuildRight (723)
: : : :- Project (717)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (716)
: : : : :- Project (711)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (710)
: : : : : :- Exchange (704)
: : : : : : +- Filter (703)
: : : : : : +- Scan parquet (702)
: : : : : +- BroadcastExchange (709)
: : : : : +- Project (708)
: : : : : +- Exchange (707)
: : : : : +- Project (706)
: : : : : +- Scan parquet (705)
: : : : +- BroadcastExchange (715)
: : : : +- Exchange (714)
: : : : +- Project (713)
: : : : +- Scan parquet (712)
: : : +- BroadcastExchange (722)
: : : +- Exchange (721)
: : : +- Project (720)
: : : +- Filter (719)
: : : +- Scan parquet (718)
: : +- BroadcastExchange (745)
: : +- Filter (744)
: : +- Scan parquet (267)
: +- Sort (802)
: +- Project (801)
: +- BroadcastHashJoin Inner BuildLeft (800)
: :- BroadcastExchange (769)
: : +- Project (768)
: : +- Filter (767)
: : +- Generate (766)
: : +- Project (765)
: : +- Filter (764)
: : +- Generate (763)
: : +- Exchange (762)
: : +- Project (761)
: : +- Filter (760)
: : +- Scan parquet (759)
: +- HashAggregate (799)
: +- Exchange (798)
: +- HashAggregate (797)
: +- Project (796)
: +- Generate (795)
: +- Project (794)
: +- Generate (793)
: +- Project (792)
: +- BroadcastHashJoin LeftOuter BuildRight (791)
: :- Project (785)
: : +- BroadcastHashJoin LeftOuter BuildRight (784)
: : :- Project (779)
: : : +- BroadcastHashJoin LeftOuter BuildRight (778)
: : : :- Exchange (772)
: : : : +- Filter (771)
: : : : +- Scan parquet (770)
: : : +- BroadcastExchange (777)
: : : +- Project (776)
: : : +- Exchange (775)
: : : +- Project (774)
: : : +- Scan parquet (773)
: : +- BroadcastExchange (783)
: : +- Exchange (782)
: : +- Project (781)
: : +- Scan parquet (780)
: +- BroadcastExchange (790)
: +- Exchange (789)
: +- Project (788)
: +- Filter (787)
: +- Scan parquet (786)
+- BroadcastExchange (811)
+- Project (810)
+- Filter (809)
+- Generate (808)
+- Exchange (807)
+- Project (806)
+- Filter (805)
+- Scan parquet (308)
+- == Initial Plan ==
Filter (837)
+- ObjectHashAggregate (836)
+- Exchange (835)
+- ObjectHashAggregate (834)
+- Project (833)
+- Window (832)
+- Sort (831)
+- Exchange (830)
+- Project (829)
+- InMemoryTableScan (3)
+- InMemoryRelation (4)
+- AdaptiveSparkPlan (814)
+- == Final Plan ==
ResultQueryStage (321)
+- Project (320)
+- * BroadcastHashJoin Inner BuildRight (319)
:- * Project (307)
: +- * BroadcastHashJoin Inner BuildLeft (306)
: :- BroadcastQueryStage (290), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (289)
: : +- AQEShuffleRead (288)
: : +- ShuffleQueryStage (287), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : +- Exchange (286)
: : +- * Project (285)
: : +- * BroadcastHashJoin LeftOuter BuildRight (284)
: : :- AQEShuffleRead (178)
: : : +- ShuffleQueryStage (177), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : : +- Exchange (176)
: : : +- * Project (175)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (174)
: : : :- * Project (42)
: : : : +- * Generate (41)
: : : : +- * Project (40)
: : : : +- * Generate (39)
: : : : +- Project (38)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (37)
: : : : :- * Project (28)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (27)
: : : : : :- * Project (19)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (18)
: : : : : : :- ShuffleQueryStage (9), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (8)
: : : : : : : +- * Filter (7)
: : : : : : : +- * ColumnarToRow (6)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastQueryStage (17), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (16)
: : : : : : +- * Project (15)
: : : : : : +- ShuffleQueryStage (14), Statistics(sizeInBytes=339.5 KiB, rowCount=2.75E+3)
: : : : : : +- Exchange (13)
: : : : : : +- * Project (12)
: : : : : : +- * ColumnarToRow (11)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastQueryStage (26), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- BroadcastExchange (25)
: : : : : +- ShuffleQueryStage (24), Statistics(sizeInBytes=4.4 KiB, rowCount=62)
: : : : : +- Exchange (23)
: : : : : +- * Project (22)
: : : : : +- * ColumnarToRow (21)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastQueryStage (36), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- BroadcastExchange (35)
: : : : +- ShuffleQueryStage (34), Statistics(sizeInBytes=92.7 KiB, rowCount=159)
: : : : +- Exchange (33)
: : : : +- * Project (32)
: : : : +- * Filter (31)
: : : : +- * ColumnarToRow (30)
: : : : +- Scan parquet (29)
: : : +- BroadcastQueryStage (173), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (172)
: : : +- * HashAggregate (171)
: : : +- AQEShuffleRead (170)
: : : +- ShuffleQueryStage (169), Statistics(sizeInBytes=27.7 KiB, rowCount=295)
: : : +- Exchange (168)
: : : +- * HashAggregate (167)
: : : +- * Project (166)
: : : +- * BroadcastHashJoin Inner BuildLeft (165)
: : : :- BroadcastQueryStage (137), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (136)
: : : : +- AQEShuffleRead (135)
: : : : +- ShuffleQueryStage (134), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : : +- Exchange (133)
: : : : +- * Project (132)
: : : : +- * BroadcastHashJoin Inner BuildLeft (131)
: : : : :- BroadcastQueryStage (109), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (108)
: : : : : +- AQEShuffleRead (107)
: : : : : +- ShuffleQueryStage (106), Statistics(sizeInBytes=43.1 KiB, rowCount=394)
: : : : : +- Exchange (105)
: : : : : +- * Project (104)
: : : : : +- * Filter (103)
: : : : : +- * Generate (102)
: : : : : +- * Filter (101)
: : : : : +- ObjectHashAggregate (100)
: : : : : +- ObjectHashAggregate (99)
: : : : : +- Generate (98)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (97)
: : : : : :- BroadcastQueryStage (50), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- BroadcastExchange (49)
: : : : : : +- ShuffleQueryStage (48), Statistics(sizeInBytes=240.1 KiB, rowCount=398)
: : : : : : +- Exchange (47)
: : : : : : +- * Project (46)
: : : : : : +- * Filter (45)
: : : : : : +- * ColumnarToRow (44)
: : : : : : +- Scan parquet (43)
: : : : : +- * Project (96)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (95)
: : : : : :- BroadcastQueryStage (63), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- BroadcastExchange (62)
: : : : : : +- * Project (61)
: : : : : : +- * Generate (60)
: : : : : : +- * Project (59)
: : : : : : +- * Filter (58)
: : : : : : +- * Generate (57)
: : : : : : +- ShuffleQueryStage (56), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : : : : : +- Exchange (55)
: : : : : : +- * Project (54)
: : : : : : +- * Filter (53)
: : : : : : +- * ColumnarToRow (52)
: : : : : : +- Scan parquet (51)
: : : : : +- * HashAggregate (94)
: : : : : +- AQEShuffleRead (93)
: : : : : +- ShuffleQueryStage (92), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- Exchange (91)
: : : : : +- * HashAggregate (90)
: : : : : +- * Project (89)
: : : : : +- * Generate (88)
: : : : : +- * Project (87)
: : : : : +- * Generate (86)
: : : : : +- Project (85)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (84)
: : : : : :- * Project (81)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (80)
: : : : : : :- * Project (72)
: : : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (71)
: : : : : : : :- ShuffleQueryStage (68), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : : : : : : : +- Exchange (67)
: : : : : : : : +- * Filter (66)
: : : : : : : : +- * ColumnarToRow (65)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastQueryStage (70), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : : +- ReusedExchange (69)
: : : : : : +- BroadcastQueryStage (79), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : : +- BroadcastExchange (78)
: : : : : : +- ShuffleQueryStage (77), Statistics(sizeInBytes=2.9 KiB, rowCount=62)
: : : : : : +- Exchange (76)
: : : : : : +- * Project (75)
: : : : : : +- * ColumnarToRow (74)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastQueryStage (83), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : : +- ReusedExchange (82)
: : : : +- * Project (130)
: : : : +- * Generate (129)
: : : : +- * Project (128)
: : : : +- * Generate (127)
: : : : +- Project (126)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (125)
: : : : :- * Project (122)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (121)
: : : : : :- * Project (118)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (117)
: : : : : : :- ShuffleQueryStage (114), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (113)
: : : : : : : +- * Filter (112)
: : : : : : : +- * ColumnarToRow (111)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastQueryStage (116), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (115)
: : : : : +- BroadcastQueryStage (120), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (119)
: : : : +- BroadcastQueryStage (124), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (123)
: : : +- AQEShuffleRead (164)
: : : +- ShuffleQueryStage (163), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- Exchange (162)
: : : +- * HashAggregate (161)
: : : +- * HashAggregate (160)
: : : +- * Project (159)
: : : +- * BroadcastHashJoin Inner BuildLeft (158)
: : : :- BroadcastQueryStage (139), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : +- ReusedExchange (138)
: : : +- * Project (157)
: : : +- * Generate (156)
: : : +- * Project (155)
: : : +- * Generate (154)
: : : +- Project (153)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (152)
: : : :- * Project (149)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (148)
: : : : :- * Project (145)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (144)
: : : : : :- ShuffleQueryStage (141), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : +- ReusedExchange (140)
: : : : : +- BroadcastQueryStage (143), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : +- ReusedExchange (142)
: : : : +- BroadcastQueryStage (147), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : +- ReusedExchange (146)
: : : +- BroadcastQueryStage (151), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : +- ReusedExchange (150)
: : +- BroadcastQueryStage (283), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (282)
: : +- AQEShuffleRead (281)
: : +- ShuffleQueryStage (280), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : +- Exchange (279)
: : +- * SerializeFromObject (278)
: : +- * MapElements (277)
: : +- * DeserializeToObject (276)
: : +- ObjectHashAggregate (275)
: : +- ObjectHashAggregate (274)
: : +- * Project (273)
: : +- * BroadcastHashJoin LeftOuter BuildRight (272)
: : :- * Project (266)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (265)
: : : :- * Project (204)
: : : : +- * Project (203)
: : : : +- * Generate (202)
: : : : +- * Generate (201)
: : : : +- * Project (200)
: : : : +- Project (199)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (198)
: : : : :- * Project (195)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (194)
: : : : : :- * BroadcastHashJoin LeftOuter BuildRight (191)
: : : : : : :- ShuffleQueryStage (182), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (181)
: : : : : : : +- * ColumnarToRow (180)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastQueryStage (190), Statistics(sizeInBytes=4.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (189)
: : : : : : +- * Project (188)
: : : : : : +- ShuffleQueryStage (187), Statistics(sizeInBytes=2.0 MiB, rowCount=2.75E+3)
: : : : : : +- Exchange (186)
: : : : : : +- * Project (185)
: : : : : : +- * ColumnarToRow (184)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastQueryStage (193), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (192)
: : : : +- BroadcastQueryStage (197), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (196)
: : : +- BroadcastQueryStage (264), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (263)
: : : +- * HashAggregate (262)
: : : +- AQEShuffleRead (261)
: : : +- ShuffleQueryStage (260), Statistics(sizeInBytes=30.0 KiB, rowCount=295)
: : : +- Exchange (259)
: : : +- * HashAggregate (258)
: : : +- * Project (257)
: : : +- * BroadcastHashJoin Inner BuildLeft (256)
: : : :- BroadcastQueryStage (252), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (251)
: : : : +- AQEShuffleRead (250)
: : : : +- ShuffleQueryStage (249), Statistics(sizeInBytes=34.6 KiB, rowCount=295)
: : : : +- Exchange (248)
: : : : +- * Project (247)
: : : : +- * BroadcastHashJoin Inner BuildLeft (246)
: : : : :- BroadcastQueryStage (227), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (226)
: : : : : +- AQEShuffleRead (225)
: : : : : +- ShuffleQueryStage (224), Statistics(sizeInBytes=46.2 KiB, rowCount=394)
: : : : : +- Exchange (223)
: : : : : +- * Project (222)
: : : : : +- * Filter (221)
: : : : : +- * Generate (220)
: : : : : +- * Filter (219)
: : : : : +- ObjectHashAggregate (218)
: : : : : +- ObjectHashAggregate (217)
: : : : : +- Generate (216)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (215)
: : : : : :- BroadcastQueryStage (206), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- ReusedExchange (205)
: : : : : +- * Project (214)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (213)
: : : : : :- BroadcastQueryStage (208), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- ReusedExchange (207)
: : : : : +- * HashAggregate (212)
: : : : : +- AQEShuffleRead (211)
: : : : : +- ShuffleQueryStage (210), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- ReusedExchange (209)
: : : : +- * Project (245)
: : : : +- * Generate (244)
: : : : +- * Project (243)
: : : : +- * Generate (242)
: : : : +- Project (241)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (240)
: : : : :- * Project (237)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (236)
: : : : : :- * Project (233)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (232)
: : : : : : :- ShuffleQueryStage (229), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- ReusedExchange (228)
: : : : : : +- BroadcastQueryStage (231), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (230)
: : : : : +- BroadcastQueryStage (235), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (234)
: : : : +- BroadcastQueryStage (239), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (238)
: : : +- AQEShuffleRead (255)
: : : +- ShuffleQueryStage (254), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- ReusedExchange (253)
: : +- BroadcastQueryStage (271), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (270)
: : +- * Filter (269)
: : +- * ColumnarToRow (268)
: : +- Scan parquet (267)
: +- * Project (305)
: +- * BroadcastHashJoin Inner BuildLeft (304)
: :- BroadcastQueryStage (300), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : +- BroadcastExchange (299)
: : +- * Project (298)
: : +- * Filter (297)
: : +- * Generate (296)
: : +- * Project (295)
: : +- * Filter (294)
: : +- * Generate (293)
: : +- ShuffleQueryStage (292), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : +- ReusedExchange (291)
: +- * HashAggregate (303)
: +- ShuffleQueryStage (302), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: +- ReusedExchange (301)
+- BroadcastQueryStage (318), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
+- BroadcastExchange (317)
+- * Project (316)
+- * Filter (315)
+- * Generate (314)
+- ShuffleQueryStage (313), Statistics(sizeInBytes=29.9 KiB, rowCount=9)
+- Exchange (312)
+- * Project (311)
+- * Filter (310)
+- * ColumnarToRow (309)
+- Scan parquet (308)
+- == Initial Plan ==
Project (813)
+- BroadcastHashJoin Inner BuildRight (812)
:- Project (804)
: +- SortMergeJoin Inner (803)
: :- Sort (758)
: : +- Exchange (757)
: : +- Project (756)
: : +- SortMergeJoin LeftOuter (755)
: : :- Sort (530)
: : : +- Exchange (529)
: : : +- Project (528)
: : : +- SortMergeJoin LeftOuter (527)
: : : :- Sort (345)
: : : : +- Project (344)
: : : : +- Generate (343)
: : : : +- Project (342)
: : : : +- Generate (341)
: : : : +- Project (340)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (339)
: : : : :- Project (334)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (333)
: : : : : :- Project (329)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (328)
: : : : : : :- Exchange (323)
: : : : : : : +- Filter (322)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastExchange (327)
: : : : : : +- Project (326)
: : : : : : +- Exchange (325)
: : : : : : +- Project (324)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastExchange (332)
: : : : : +- Exchange (331)
: : : : : +- Project (330)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastExchange (338)
: : : : +- Exchange (337)
: : : : +- Project (336)
: : : : +- Filter (335)
: : : : +- Scan parquet (29)
: : : +- Sort (526)
: : : +- HashAggregate (525)
: : : +- Exchange (524)
: : : +- HashAggregate (523)
: : : +- Project (522)
: : : +- SortMergeJoin Inner (521)
: : : :- Sort (429)
: : : : +- Exchange (428)
: : : : +- Project (427)
: : : : +- SortMergeJoin Inner (426)
: : : : :- Sort (398)
: : : : : +- Exchange (397)
: : : : : +- Project (396)
: : : : : +- Filter (395)
: : : : : +- Generate (394)
: : : : : +- Filter (393)
: : : : : +- ObjectHashAggregate (392)
: : : : : +- ObjectHashAggregate (391)
: : : : : +- Generate (390)
: : : : : +- BroadcastHashJoin Inner BuildLeft (389)
: : : : : :- BroadcastExchange (349)
: : : : : : +- Exchange (348)
: : : : : : +- Project (347)
: : : : : : +- Filter (346)
: : : : : : +- Scan parquet (43)
: : : : : +- Project (388)
: : : : : +- BroadcastHashJoin Inner BuildLeft (387)
: : : : : :- BroadcastExchange (358)
: : : : : : +- Project (357)
: : : : : : +- Generate (356)
: : : : : : +- Project (355)
: : : : : : +- Filter (354)
: : : : : : +- Generate (353)
: : : : : : +- Exchange (352)
: : : : : : +- Project (351)
: : : : : : +- Filter (350)
: : : : : : +- Scan parquet (51)
: : : : : +- HashAggregate (386)
: : : : : +- Exchange (385)
: : : : : +- HashAggregate (384)
: : : : : +- Project (383)
: : : : : +- Generate (382)
: : : : : +- Project (381)
: : : : : +- Generate (380)
: : : : : +- Project (379)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (378)
: : : : : :- Project (372)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (371)
: : : : : : :- Project (367)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (366)
: : : : : : : :- Exchange (360)
: : : : : : : : +- Filter (359)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastExchange (365)
: : : : : : : +- Project (364)
: : : : : : : +- Exchange (363)
: : : : : : : +- Project (362)
: : : : : : : +- Scan parquet (361)
: : : : : : +- BroadcastExchange (370)
: : : : : : +- Exchange (369)
: : : : : : +- Project (368)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastExchange (377)
: : : : : +- Exchange (376)
: : : : : +- Project (375)
: : : : : +- Filter (374)
: : : : : +- Scan parquet (373)
: : : : +- Sort (425)
: : : : +- Project (424)
: : : : +- Generate (423)
: : : : +- Project (422)
: : : : +- Generate (421)
: : : : +- Project (420)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (419)
: : : : :- Project (413)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (412)
: : : : : :- Project (407)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (406)
: : : : : : :- Exchange (400)
: : : : : : : +- Filter (399)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastExchange (405)
: : : : : : +- Project (404)
: : : : : : +- Exchange (403)
: : : : : : +- Project (402)
: : : : : : +- Scan parquet (401)
: : : : : +- BroadcastExchange (411)
: : : : : +- Exchange (410)
: : : : : +- Project (409)
: : : : : +- Scan parquet (408)
: : : : +- BroadcastExchange (418)
: : : : +- Exchange (417)
: : : : +- Project (416)
: : : : +- Filter (415)
: : : : +- Scan parquet (414)
: : : +- Sort (520)
: : : +- Exchange (519)
: : : +- HashAggregate (518)
: : : +- HashAggregate (517)
: : : +- Project (516)
: : : +- SortMergeJoin Inner (515)
: : : :- Sort (486)
: : : : +- Exchange (485)
: : : : +- Project (484)
: : : : +- Filter (483)
: : : : +- Generate (482)
: : : : +- Filter (481)
: : : : +- ObjectHashAggregate (480)
: : : : +- ObjectHashAggregate (479)
: : : : +- Generate (478)
: : : : +- BroadcastHashJoin Inner BuildLeft (477)
: : : : :- BroadcastExchange (434)
: : : : : +- Exchange (433)
: : : : : +- Project (432)
: : : : : +- Filter (431)
: : : : : +- Scan parquet (430)
: : : : +- Project (476)
: : : : +- BroadcastHashJoin Inner BuildLeft (475)
: : : : :- BroadcastExchange (444)
: : : : : +- Project (443)
: : : : : +- Generate (442)
: : : : : +- Project (441)
: : : : : +- Filter (440)
: : : : : +- Generate (439)
: : : : : +- Exchange (438)
: : : : : +- Project (437)
: : : : : +- Filter (436)
: : : : : +- Scan parquet (435)
: : : : +- HashAggregate (474)
: : : : +- Exchange (473)
: : : : +- HashAggregate (472)
: : : : +- Project (471)
: : : : +- Generate (470)
: : : : +- Project (469)
: : : : +- Generate (468)
: : : : +- Project (467)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (466)
: : : : :- Project (460)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (459)
: : : : : :- Project (454)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (453)
: : : : : : :- Exchange (447)
: : : : : : : +- Filter (446)
: : : : : : : +- Scan parquet (445)
: : : : : : +- BroadcastExchange (452)
: : : : : : +- Project (451)
: : : : : : +- Exchange (450)
: : : : : : +- Project (449)
: : : : : : +- Scan parquet (448)
: : : : : +- BroadcastExchange (458)
: : : : : +- Exchange (457)
: : : : : +- Project (456)
: : : : : +- Scan parquet (455)
: : : : +- BroadcastExchange (465)
: : : : +- Exchange (464)
: : : : +- Project (463)
: : : : +- Filter (462)
: : : : +- Scan parquet (461)
: : : +- Sort (514)
: : : +- Project (513)
: : : +- Generate (512)
: : : +- Project (511)
: : : +- Generate (510)
: : : +- Project (509)
: : : +- BroadcastHashJoin LeftOuter BuildRight (508)
: : : :- Project (502)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (501)
: : : : :- Project (496)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (495)
: : : : : :- Exchange (489)
: : : : : : +- Filter (488)
: : : : : : +- Scan parquet (487)
: : : : : +- BroadcastExchange (494)
: : : : : +- Project (493)
: : : : : +- Exchange (492)
: : : : : +- Project (491)
: : : : : +- Scan parquet (490)
: : : : +- BroadcastExchange (500)
: : : : +- Exchange (499)
: : : : +- Project (498)
: : : : +- Scan parquet (497)
: : : +- BroadcastExchange (507)
: : : +- Exchange (506)
: : : +- Project (505)
: : : +- Filter (504)
: : : +- Scan parquet (503)
: : +- Sort (754)
: : +- Exchange (753)
: : +- SerializeFromObject (752)
: : +- MapElements (751)
: : +- DeserializeToObject (750)
: : +- ObjectHashAggregate (749)
: : +- ObjectHashAggregate (748)
: : +- Project (747)
: : +- BroadcastHashJoin LeftOuter BuildRight (746)
: : :- Project (743)
: : : +- SortMergeJoin LeftOuter (742)
: : : :- Sort (555)
: : : : +- Project (554)
: : : : +- Project (553)
: : : : +- Generate (552)
: : : : +- Generate (551)
: : : : +- Project (550)
: : : : +- Project (549)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (548)
: : : : :- Project (542)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (541)
: : : : : :- BroadcastHashJoin LeftOuter BuildRight (536)
: : : : : : :- Exchange (531)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastExchange (535)
: : : : : : +- Project (534)
: : : : : : +- Exchange (533)
: : : : : : +- Project (532)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastExchange (540)
: : : : : +- Exchange (539)
: : : : : +- Project (538)
: : : : : +- Scan parquet (537)
: : : : +- BroadcastExchange (547)
: : : : +- Exchange (546)
: : : : +- Project (545)
: : : : +- Filter (544)
: : : : +- Scan parquet (543)
: : : +- Sort (741)
: : : +- HashAggregate (740)
: : : +- Exchange (739)
: : : +- HashAggregate (738)
: : : +- Project (737)
: : : +- SortMergeJoin Inner (736)
: : : :- Sort (644)
: : : : +- Exchange (643)
: : : : +- Project (642)
: : : : +- SortMergeJoin Inner (641)
: : : : :- Sort (612)
: : : : : +- Exchange (611)
: : : : : +- Project (610)
: : : : : +- Filter (609)
: : : : : +- Generate (608)
: : : : : +- Filter (607)
: : : : : +- ObjectHashAggregate (606)
: : : : : +- ObjectHashAggregate (605)
: : : : : +- Generate (604)
: : : : : +- BroadcastHashJoin Inner BuildLeft (603)
: : : : : :- BroadcastExchange (560)
: : : : : : +- Exchange (559)
: : : : : : +- Project (558)
: : : : : : +- Filter (557)
: : : : : : +- Scan parquet (556)
: : : : : +- Project (602)
: : : : : +- BroadcastHashJoin Inner BuildLeft (601)
: : : : : :- BroadcastExchange (570)
: : : : : : +- Project (569)
: : : : : : +- Generate (568)
: : : : : : +- Project (567)
: : : : : : +- Filter (566)
: : : : : : +- Generate (565)
: : : : : : +- Exchange (564)
: : : : : : +- Project (563)
: : : : : : +- Filter (562)
: : : : : : +- Scan parquet (561)
: : : : : +- HashAggregate (600)
: : : : : +- Exchange (599)
: : : : : +- HashAggregate (598)
: : : : : +- Project (597)
: : : : : +- Generate (596)
: : : : : +- Project (595)
: : : : : +- Generate (594)
: : : : : +- Project (593)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (592)
: : : : : :- Project (586)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (585)
: : : : : : :- Project (580)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (579)
: : : : : : : :- Exchange (573)
: : : : : : : : +- Filter (572)
: : : : : : : : +- Scan parquet (571)
: : : : : : : +- BroadcastExchange (578)
: : : : : : : +- Project (577)
: : : : : : : +- Exchange (576)
: : : : : : : +- Project (575)
: : : : : : : +- Scan parquet (574)
: : : : : : +- BroadcastExchange (584)
: : : : : : +- Exchange (583)
: : : : : : +- Project (582)
: : : : : : +- Scan parquet (581)
: : : : : +- BroadcastExchange (591)
: : : : : +- Exchange (590)
: : : : : +- Project (589)
: : : : : +- Filter (588)
: : : : : +- Scan parquet (587)
: : : : +- Sort (640)
: : : : +- Project (639)
: : : : +- Generate (638)
: : : : +- Project (637)
: : : : +- Generate (636)
: : : : +- Project (635)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (634)
: : : : :- Project (628)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (627)
: : : : : :- Project (622)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (621)
: : : : : : :- Exchange (615)
: : : : : : : +- Filter (614)
: : : : : : : +- Scan parquet (613)
: : : : : : +- BroadcastExchange (620)
: : : : : : +- Project (619)
: : : : : : +- Exchange (618)
: : : : : : +- Project (617)
: : : : : : +- Scan parquet (616)
: : : : : +- BroadcastExchange (626)
: : : : : +- Exchange (625)
: : : : : +- Project (624)
: : : : : +- Scan parquet (623)
: : : : +- BroadcastExchange (633)
: : : : +- Exchange (632)
: : : : +- Project (631)
: : : : +- Filter (630)
: : : : +- Scan parquet (629)
: : : +- Sort (735)
: : : +- Exchange (734)
: : : +- HashAggregate (733)
: : : +- HashAggregate (732)
: : : +- Project (731)
: : : +- SortMergeJoin Inner (730)
: : : :- Sort (701)
: : : : +- Exchange (700)
: : : : +- Project (699)
: : : : +- Filter (698)
: : : : +- Generate (697)
: : : : +- Filter (696)
: : : : +- ObjectHashAggregate (695)
: : : : +- ObjectHashAggregate (694)
: : : : +- Generate (693)
: : : : +- BroadcastHashJoin Inner BuildLeft (692)
: : : : :- BroadcastExchange (649)
: : : : : +- Exchange (648)
: : : : : +- Project (647)
: : : : : +- Filter (646)
: : : : : +- Scan parquet (645)
: : : : +- Project (691)
: : : : +- BroadcastHashJoin Inner BuildLeft (690)
: : : : :- BroadcastExchange (659)
: : : : : +- Project (658)
: : : : : +- Generate (657)
: : : : : +- Project (656)
: : : : : +- Filter (655)
: : : : : +- Generate (654)
: : : : : +- Exchange (653)
: : : : : +- Project (652)
: : : : : +- Filter (651)
: : : : : +- Scan parquet (650)
: : : : +- HashAggregate (689)
: : : : +- Exchange (688)
: : : : +- HashAggregate (687)
: : : : +- Project (686)
: : : : +- Generate (685)
: : : : +- Project (684)
: : : : +- Generate (683)
: : : : +- Project (682)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (681)
: : : : :- Project (675)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (674)
: : : : : :- Project (669)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (668)
: : : : : : :- Exchange (662)
: : : : : : : +- Filter (661)
: : : : : : : +- Scan parquet (660)
: : : : : : +- BroadcastExchange (667)
: : : : : : +- Project (666)
: : : : : : +- Exchange (665)
: : : : : : +- Project (664)
: : : : : : +- Scan parquet (663)
: : : : : +- BroadcastExchange (673)
: : : : : +- Exchange (672)
: : : : : +- Project (671)
: : : : : +- Scan parquet (670)
: : : : +- BroadcastExchange (680)
: : : : +- Exchange (679)
: : : : +- Project (678)
: : : : +- Filter (677)
: : : : +- Scan parquet (676)
: : : +- Sort (729)
: : : +- Project (728)
: : : +- Generate (727)
: : : +- Project (726)
: : : +- Generate (725)
: : : +- Project (724)
: : : +- BroadcastHashJoin LeftOuter BuildRight (723)
: : : :- Project (717)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (716)
: : : : :- Project (711)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (710)
: : : : : :- Exchange (704)
: : : : : : +- Filter (703)
: : : : : : +- Scan parquet (702)
: : : : : +- BroadcastExchange (709)
: : : : : +- Project (708)
: : : : : +- Exchange (707)
: : : : : +- Project (706)
: : : : : +- Scan parquet (705)
: : : : +- BroadcastExchange (715)
: : : : +- Exchange (714)
: : : : +- Project (713)
: : : : +- Scan parquet (712)
: : : +- BroadcastExchange (722)
: : : +- Exchange (721)
: : : +- Project (720)
: : : +- Filter (719)
: : : +- Scan parquet (718)
: : +- BroadcastExchange (745)
: : +- Filter (744)
: : +- Scan parquet (267)
: +- Sort (802)
: +- Project (801)
: +- BroadcastHashJoin Inner BuildLeft (800)
: :- BroadcastExchange (769)
: : +- Project (768)
: : +- Filter (767)
: : +- Generate (766)
: : +- Project (765)
: : +- Filter (764)
: : +- Generate (763)
: : +- Exchange (762)
: : +- Project (761)
: : +- Filter (760)
: : +- Scan parquet (759)
: +- HashAggregate (799)
: +- Exchange (798)
: +- HashAggregate (797)
: +- Project (796)
: +- Generate (795)
: +- Project (794)
: +- Generate (793)
: +- Project (792)
: +- BroadcastHashJoin LeftOuter BuildRight (791)
: :- Project (785)
: : +- BroadcastHashJoin LeftOuter BuildRight (784)
: : :- Project (779)
: : : +- BroadcastHashJoin LeftOuter BuildRight (778)
: : : :- Exchange (772)
: : : : +- Filter (771)
: : : : +- Scan parquet (770)
: : : +- BroadcastExchange (777)
: : : +- Project (776)
: : : +- Exchange (775)
: : : +- Project (774)
: : : +- Scan parquet (773)
: : +- BroadcastExchange (783)
: : +- Exchange (782)
: : +- Project (781)
: : +- Scan parquet (780)
: +- BroadcastExchange (790)
: +- Exchange (789)
: +- Project (788)
: +- Filter (787)
: +- Scan parquet (786)
+- BroadcastExchange (811)
+- Project (810)
+- Filter (809)
+- Generate (808)
+- Exchange (807)
+- Project (806)
+- Filter (805)
+- Scan parquet (308)
+- == Initial Plan ==
Execute InsertIntoHadoopFsRelationCommand (848)
+- WriteFiles (847)
+- Project (846)
+- Filter (845)
+- InMemoryTableScan (1)
+- InMemoryRelation (2)
+- AdaptiveSparkPlan (838)
+- == Final Plan ==
ResultQueryStage (828)
+- * Filter (827)
+- ObjectHashAggregate (826)
+- ShuffleQueryStage (825), Statistics(sizeInBytes=13.0 KiB, rowCount=41)
+- Exchange (824)
+- ObjectHashAggregate (823)
+- Project (822)
+- Window (821)
+- * Sort (820)
+- AQEShuffleRead (819)
+- ShuffleQueryStage (818), Statistics(sizeInBytes=17.7 KiB, rowCount=41)
+- Exchange (817)
+- * Project (816)
+- TableCacheQueryStage (815), Statistics(sizeInBytes=2.73E+162 B)
+- InMemoryTableScan (3)
+- InMemoryRelation (4)
+- AdaptiveSparkPlan (814)
+- == Final Plan ==
ResultQueryStage (321)
+- Project (320)
+- * BroadcastHashJoin Inner BuildRight (319)
:- * Project (307)
: +- * BroadcastHashJoin Inner BuildLeft (306)
: :- BroadcastQueryStage (290), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (289)
: : +- AQEShuffleRead (288)
: : +- ShuffleQueryStage (287), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : +- Exchange (286)
: : +- * Project (285)
: : +- * BroadcastHashJoin LeftOuter BuildRight (284)
: : :- AQEShuffleRead (178)
: : : +- ShuffleQueryStage (177), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : : +- Exchange (176)
: : : +- * Project (175)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (174)
: : : :- * Project (42)
: : : : +- * Generate (41)
: : : : +- * Project (40)
: : : : +- * Generate (39)
: : : : +- Project (38)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (37)
: : : : :- * Project (28)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (27)
: : : : : :- * Project (19)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (18)
: : : : : : :- ShuffleQueryStage (9), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (8)
: : : : : : : +- * Filter (7)
: : : : : : : +- * ColumnarToRow (6)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastQueryStage (17), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (16)
: : : : : : +- * Project (15)
: : : : : : +- ShuffleQueryStage (14), Statistics(sizeInBytes=339.5 KiB, rowCount=2.75E+3)
: : : : : : +- Exchange (13)
: : : : : : +- * Project (12)
: : : : : : +- * ColumnarToRow (11)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastQueryStage (26), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- BroadcastExchange (25)
: : : : : +- ShuffleQueryStage (24), Statistics(sizeInBytes=4.4 KiB, rowCount=62)
: : : : : +- Exchange (23)
: : : : : +- * Project (22)
: : : : : +- * ColumnarToRow (21)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastQueryStage (36), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- BroadcastExchange (35)
: : : : +- ShuffleQueryStage (34), Statistics(sizeInBytes=92.7 KiB, rowCount=159)
: : : : +- Exchange (33)
: : : : +- * Project (32)
: : : : +- * Filter (31)
: : : : +- * ColumnarToRow (30)
: : : : +- Scan parquet (29)
: : : +- BroadcastQueryStage (173), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (172)
: : : +- * HashAggregate (171)
: : : +- AQEShuffleRead (170)
: : : +- ShuffleQueryStage (169), Statistics(sizeInBytes=27.7 KiB, rowCount=295)
: : : +- Exchange (168)
: : : +- * HashAggregate (167)
: : : +- * Project (166)
: : : +- * BroadcastHashJoin Inner BuildLeft (165)
: : : :- BroadcastQueryStage (137), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (136)
: : : : +- AQEShuffleRead (135)
: : : : +- ShuffleQueryStage (134), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : : +- Exchange (133)
: : : : +- * Project (132)
: : : : +- * BroadcastHashJoin Inner BuildLeft (131)
: : : : :- BroadcastQueryStage (109), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (108)
: : : : : +- AQEShuffleRead (107)
: : : : : +- ShuffleQueryStage (106), Statistics(sizeInBytes=43.1 KiB, rowCount=394)
: : : : : +- Exchange (105)
: : : : : +- * Project (104)
: : : : : +- * Filter (103)
: : : : : +- * Generate (102)
: : : : : +- * Filter (101)
: : : : : +- ObjectHashAggregate (100)
: : : : : +- ObjectHashAggregate (99)
: : : : : +- Generate (98)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (97)
: : : : : :- BroadcastQueryStage (50), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- BroadcastExchange (49)
: : : : : : +- ShuffleQueryStage (48), Statistics(sizeInBytes=240.1 KiB, rowCount=398)
: : : : : : +- Exchange (47)
: : : : : : +- * Project (46)
: : : : : : +- * Filter (45)
: : : : : : +- * ColumnarToRow (44)
: : : : : : +- Scan parquet (43)
: : : : : +- * Project (96)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (95)
: : : : : :- BroadcastQueryStage (63), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- BroadcastExchange (62)
: : : : : : +- * Project (61)
: : : : : : +- * Generate (60)
: : : : : : +- * Project (59)
: : : : : : +- * Filter (58)
: : : : : : +- * Generate (57)
: : : : : : +- ShuffleQueryStage (56), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : : : : : +- Exchange (55)
: : : : : : +- * Project (54)
: : : : : : +- * Filter (53)
: : : : : : +- * ColumnarToRow (52)
: : : : : : +- Scan parquet (51)
: : : : : +- * HashAggregate (94)
: : : : : +- AQEShuffleRead (93)
: : : : : +- ShuffleQueryStage (92), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- Exchange (91)
: : : : : +- * HashAggregate (90)
: : : : : +- * Project (89)
: : : : : +- * Generate (88)
: : : : : +- * Project (87)
: : : : : +- * Generate (86)
: : : : : +- Project (85)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (84)
: : : : : :- * Project (81)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (80)
: : : : : : :- * Project (72)
: : : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (71)
: : : : : : : :- ShuffleQueryStage (68), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : : : : : : : +- Exchange (67)
: : : : : : : : +- * Filter (66)
: : : : : : : : +- * ColumnarToRow (65)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastQueryStage (70), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : : +- ReusedExchange (69)
: : : : : : +- BroadcastQueryStage (79), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : : +- BroadcastExchange (78)
: : : : : : +- ShuffleQueryStage (77), Statistics(sizeInBytes=2.9 KiB, rowCount=62)
: : : : : : +- Exchange (76)
: : : : : : +- * Project (75)
: : : : : : +- * ColumnarToRow (74)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastQueryStage (83), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : : +- ReusedExchange (82)
: : : : +- * Project (130)
: : : : +- * Generate (129)
: : : : +- * Project (128)
: : : : +- * Generate (127)
: : : : +- Project (126)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (125)
: : : : :- * Project (122)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (121)
: : : : : :- * Project (118)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (117)
: : : : : : :- ShuffleQueryStage (114), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (113)
: : : : : : : +- * Filter (112)
: : : : : : : +- * ColumnarToRow (111)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastQueryStage (116), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (115)
: : : : : +- BroadcastQueryStage (120), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (119)
: : : : +- BroadcastQueryStage (124), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (123)
: : : +- AQEShuffleRead (164)
: : : +- ShuffleQueryStage (163), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- Exchange (162)
: : : +- * HashAggregate (161)
: : : +- * HashAggregate (160)
: : : +- * Project (159)
: : : +- * BroadcastHashJoin Inner BuildLeft (158)
: : : :- BroadcastQueryStage (139), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : +- ReusedExchange (138)
: : : +- * Project (157)
: : : +- * Generate (156)
: : : +- * Project (155)
: : : +- * Generate (154)
: : : +- Project (153)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (152)
: : : :- * Project (149)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (148)
: : : : :- * Project (145)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (144)
: : : : : :- ShuffleQueryStage (141), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : +- ReusedExchange (140)
: : : : : +- BroadcastQueryStage (143), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : +- ReusedExchange (142)
: : : : +- BroadcastQueryStage (147), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : +- ReusedExchange (146)
: : : +- BroadcastQueryStage (151), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : +- ReusedExchange (150)
: : +- BroadcastQueryStage (283), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (282)
: : +- AQEShuffleRead (281)
: : +- ShuffleQueryStage (280), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : +- Exchange (279)
: : +- * SerializeFromObject (278)
: : +- * MapElements (277)
: : +- * DeserializeToObject (276)
: : +- ObjectHashAggregate (275)
: : +- ObjectHashAggregate (274)
: : +- * Project (273)
: : +- * BroadcastHashJoin LeftOuter BuildRight (272)
: : :- * Project (266)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (265)
: : : :- * Project (204)
: : : : +- * Project (203)
: : : : +- * Generate (202)
: : : : +- * Generate (201)
: : : : +- * Project (200)
: : : : +- Project (199)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (198)
: : : : :- * Project (195)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (194)
: : : : : :- * BroadcastHashJoin LeftOuter BuildRight (191)
: : : : : : :- ShuffleQueryStage (182), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (181)
: : : : : : : +- * ColumnarToRow (180)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastQueryStage (190), Statistics(sizeInBytes=4.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (189)
: : : : : : +- * Project (188)
: : : : : : +- ShuffleQueryStage (187), Statistics(sizeInBytes=2.0 MiB, rowCount=2.75E+3)
: : : : : : +- Exchange (186)
: : : : : : +- * Project (185)
: : : : : : +- * ColumnarToRow (184)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastQueryStage (193), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (192)
: : : : +- BroadcastQueryStage (197), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (196)
: : : +- BroadcastQueryStage (264), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (263)
: : : +- * HashAggregate (262)
: : : +- AQEShuffleRead (261)
: : : +- ShuffleQueryStage (260), Statistics(sizeInBytes=30.0 KiB, rowCount=295)
: : : +- Exchange (259)
: : : +- * HashAggregate (258)
: : : +- * Project (257)
: : : +- * BroadcastHashJoin Inner BuildLeft (256)
: : : :- BroadcastQueryStage (252), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (251)
: : : : +- AQEShuffleRead (250)
: : : : +- ShuffleQueryStage (249), Statistics(sizeInBytes=34.6 KiB, rowCount=295)
: : : : +- Exchange (248)
: : : : +- * Project (247)
: : : : +- * BroadcastHashJoin Inner BuildLeft (246)
: : : : :- BroadcastQueryStage (227), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (226)
: : : : : +- AQEShuffleRead (225)
: : : : : +- ShuffleQueryStage (224), Statistics(sizeInBytes=46.2 KiB, rowCount=394)
: : : : : +- Exchange (223)
: : : : : +- * Project (222)
: : : : : +- * Filter (221)
: : : : : +- * Generate (220)
: : : : : +- * Filter (219)
: : : : : +- ObjectHashAggregate (218)
: : : : : +- ObjectHashAggregate (217)
: : : : : +- Generate (216)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (215)
: : : : : :- BroadcastQueryStage (206), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- ReusedExchange (205)
: : : : : +- * Project (214)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (213)
: : : : : :- BroadcastQueryStage (208), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- ReusedExchange (207)
: : : : : +- * HashAggregate (212)
: : : : : +- AQEShuffleRead (211)
: : : : : +- ShuffleQueryStage (210), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- ReusedExchange (209)
: : : : +- * Project (245)
: : : : +- * Generate (244)
: : : : +- * Project (243)
: : : : +- * Generate (242)
: : : : +- Project (241)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (240)
: : : : :- * Project (237)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (236)
: : : : : :- * Project (233)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (232)
: : : : : : :- ShuffleQueryStage (229), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- ReusedExchange (228)
: : : : : : +- BroadcastQueryStage (231), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (230)
: : : : : +- BroadcastQueryStage (235), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (234)
: : : : +- BroadcastQueryStage (239), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (238)
: : : +- AQEShuffleRead (255)
: : : +- ShuffleQueryStage (254), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- ReusedExchange (253)
: : +- BroadcastQueryStage (271), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (270)
: : +- * Filter (269)
: : +- * ColumnarToRow (268)
: : +- Scan parquet (267)
: +- * Project (305)
: +- * BroadcastHashJoin Inner BuildLeft (304)
: :- BroadcastQueryStage (300), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : +- BroadcastExchange (299)
: : +- * Project (298)
: : +- * Filter (297)
: : +- * Generate (296)
: : +- * Project (295)
: : +- * Filter (294)
: : +- * Generate (293)
: : +- ShuffleQueryStage (292), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : +- ReusedExchange (291)
: +- * HashAggregate (303)
: +- ShuffleQueryStage (302), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: +- ReusedExchange (301)
+- BroadcastQueryStage (318), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
+- BroadcastExchange (317)
+- * Project (316)
+- * Filter (315)
+- * Generate (314)
+- ShuffleQueryStage (313), Statistics(sizeInBytes=29.9 KiB, rowCount=9)
+- Exchange (312)
+- * Project (311)
+- * Filter (310)
+- * ColumnarToRow (309)
+- Scan parquet (308)
+- == Initial Plan ==
Project (813)
+- BroadcastHashJoin Inner BuildRight (812)
:- Project (804)
: +- SortMergeJoin Inner (803)
: :- Sort (758)
: : +- Exchange (757)
: : +- Project (756)
: : +- SortMergeJoin LeftOuter (755)
: : :- Sort (530)
: : : +- Exchange (529)
: : : +- Project (528)
: : : +- SortMergeJoin LeftOuter (527)
: : : :- Sort (345)
: : : : +- Project (344)
: : : : +- Generate (343)
: : : : +- Project (342)
: : : : +- Generate (341)
: : : : +- Project (340)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (339)
: : : : :- Project (334)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (333)
: : : : : :- Project (329)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (328)
: : : : : : :- Exchange (323)
: : : : : : : +- Filter (322)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastExchange (327)
: : : : : : +- Project (326)
: : : : : : +- Exchange (325)
: : : : : : +- Project (324)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastExchange (332)
: : : : : +- Exchange (331)
: : : : : +- Project (330)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastExchange (338)
: : : : +- Exchange (337)
: : : : +- Project (336)
: : : : +- Filter (335)
: : : : +- Scan parquet (29)
: : : +- Sort (526)
: : : +- HashAggregate (525)
: : : +- Exchange (524)
: : : +- HashAggregate (523)
: : : +- Project (522)
: : : +- SortMergeJoin Inner (521)
: : : :- Sort (429)
: : : : +- Exchange (428)
: : : : +- Project (427)
: : : : +- SortMergeJoin Inner (426)
: : : : :- Sort (398)
: : : : : +- Exchange (397)
: : : : : +- Project (396)
: : : : : +- Filter (395)
: : : : : +- Generate (394)
: : : : : +- Filter (393)
: : : : : +- ObjectHashAggregate (392)
: : : : : +- ObjectHashAggregate (391)
: : : : : +- Generate (390)
: : : : : +- BroadcastHashJoin Inner BuildLeft (389)
: : : : : :- BroadcastExchange (349)
: : : : : : +- Exchange (348)
: : : : : : +- Project (347)
: : : : : : +- Filter (346)
: : : : : : +- Scan parquet (43)
: : : : : +- Project (388)
: : : : : +- BroadcastHashJoin Inner BuildLeft (387)
: : : : : :- BroadcastExchange (358)
: : : : : : +- Project (357)
: : : : : : +- Generate (356)
: : : : : : +- Project (355)
: : : : : : +- Filter (354)
: : : : : : +- Generate (353)
: : : : : : +- Exchange (352)
: : : : : : +- Project (351)
: : : : : : +- Filter (350)
: : : : : : +- Scan parquet (51)
: : : : : +- HashAggregate (386)
: : : : : +- Exchange (385)
: : : : : +- HashAggregate (384)
: : : : : +- Project (383)
: : : : : +- Generate (382)
: : : : : +- Project (381)
: : : : : +- Generate (380)
: : : : : +- Project (379)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (378)
: : : : : :- Project (372)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (371)
: : : : : : :- Project (367)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (366)
: : : : : : : :- Exchange (360)
: : : : : : : : +- Filter (359)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastExchange (365)
: : : : : : : +- Project (364)
: : : : : : : +- Exchange (363)
: : : : : : : +- Project (362)
: : : : : : : +- Scan parquet (361)
: : : : : : +- BroadcastExchange (370)
: : : : : : +- Exchange (369)
: : : : : : +- Project (368)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastExchange (377)
: : : : : +- Exchange (376)
: : : : : +- Project (375)
: : : : : +- Filter (374)
: : : : : +- Scan parquet (373)
: : : : +- Sort (425)
: : : : +- Project (424)
: : : : +- Generate (423)
: : : : +- Project (422)
: : : : +- Generate (421)
: : : : +- Project (420)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (419)
: : : : :- Project (413)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (412)
: : : : : :- Project (407)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (406)
: : : : : : :- Exchange (400)
: : : : : : : +- Filter (399)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastExchange (405)
: : : : : : +- Project (404)
: : : : : : +- Exchange (403)
: : : : : : +- Project (402)
: : : : : : +- Scan parquet (401)
: : : : : +- BroadcastExchange (411)
: : : : : +- Exchange (410)
: : : : : +- Project (409)
: : : : : +- Scan parquet (408)
: : : : +- BroadcastExchange (418)
: : : : +- Exchange (417)
: : : : +- Project (416)
: : : : +- Filter (415)
: : : : +- Scan parquet (414)
: : : +- Sort (520)
: : : +- Exchange (519)
: : : +- HashAggregate (518)
: : : +- HashAggregate (517)
: : : +- Project (516)
: : : +- SortMergeJoin Inner (515)
: : : :- Sort (486)
: : : : +- Exchange (485)
: : : : +- Project (484)
: : : : +- Filter (483)
: : : : +- Generate (482)
: : : : +- Filter (481)
: : : : +- ObjectHashAggregate (480)
: : : : +- ObjectHashAggregate (479)
: : : : +- Generate (478)
: : : : +- BroadcastHashJoin Inner BuildLeft (477)
: : : : :- BroadcastExchange (434)
: : : : : +- Exchange (433)
: : : : : +- Project (432)
: : : : : +- Filter (431)
: : : : : +- Scan parquet (430)
: : : : +- Project (476)
: : : : +- BroadcastHashJoin Inner BuildLeft (475)
: : : : :- BroadcastExchange (444)
: : : : : +- Project (443)
: : : : : +- Generate (442)
: : : : : +- Project (441)
: : : : : +- Filter (440)
: : : : : +- Generate (439)
: : : : : +- Exchange (438)
: : : : : +- Project (437)
: : : : : +- Filter (436)
: : : : : +- Scan parquet (435)
: : : : +- HashAggregate (474)
: : : : +- Exchange (473)
: : : : +- HashAggregate (472)
: : : : +- Project (471)
: : : : +- Generate (470)
: : : : +- Project (469)
: : : : +- Generate (468)
: : : : +- Project (467)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (466)
: : : : :- Project (460)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (459)
: : : : : :- Project (454)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (453)
: : : : : : :- Exchange (447)
: : : : : : : +- Filter (446)
: : : : : : : +- Scan parquet (445)
: : : : : : +- BroadcastExchange (452)
: : : : : : +- Project (451)
: : : : : : +- Exchange (450)
: : : : : : +- Project (449)
: : : : : : +- Scan parquet (448)
: : : : : +- BroadcastExchange (458)
: : : : : +- Exchange (457)
: : : : : +- Project (456)
: : : : : +- Scan parquet (455)
: : : : +- BroadcastExchange (465)
: : : : +- Exchange (464)
: : : : +- Project (463)
: : : : +- Filter (462)
: : : : +- Scan parquet (461)
: : : +- Sort (514)
: : : +- Project (513)
: : : +- Generate (512)
: : : +- Project (511)
: : : +- Generate (510)
: : : +- Project (509)
: : : +- BroadcastHashJoin LeftOuter BuildRight (508)
: : : :- Project (502)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (501)
: : : : :- Project (496)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (495)
: : : : : :- Exchange (489)
: : : : : : +- Filter (488)
: : : : : : +- Scan parquet (487)
: : : : : +- BroadcastExchange (494)
: : : : : +- Project (493)
: : : : : +- Exchange (492)
: : : : : +- Project (491)
: : : : : +- Scan parquet (490)
: : : : +- BroadcastExchange (500)
: : : : +- Exchange (499)
: : : : +- Project (498)
: : : : +- Scan parquet (497)
: : : +- BroadcastExchange (507)
: : : +- Exchange (506)
: : : +- Project (505)
: : : +- Filter (504)
: : : +- Scan parquet (503)
: : +- Sort (754)
: : +- Exchange (753)
: : +- SerializeFromObject (752)
: : +- MapElements (751)
: : +- DeserializeToObject (750)
: : +- ObjectHashAggregate (749)
: : +- ObjectHashAggregate (748)
: : +- Project (747)
: : +- BroadcastHashJoin LeftOuter BuildRight (746)
: : :- Project (743)
: : : +- SortMergeJoin LeftOuter (742)
: : : :- Sort (555)
: : : : +- Project (554)
: : : : +- Project (553)
: : : : +- Generate (552)
: : : : +- Generate (551)
: : : : +- Project (550)
: : : : +- Project (549)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (548)
: : : : :- Project (542)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (541)
: : : : : :- BroadcastHashJoin LeftOuter BuildRight (536)
: : : : : : :- Exchange (531)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastExchange (535)
: : : : : : +- Project (534)
: : : : : : +- Exchange (533)
: : : : : : +- Project (532)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastExchange (540)
: : : : : +- Exchange (539)
: : : : : +- Project (538)
: : : : : +- Scan parquet (537)
: : : : +- BroadcastExchange (547)
: : : : +- Exchange (546)
: : : : +- Project (545)
: : : : +- Filter (544)
: : : : +- Scan parquet (543)
: : : +- Sort (741)
: : : +- HashAggregate (740)
: : : +- Exchange (739)
: : : +- HashAggregate (738)
: : : +- Project (737)
: : : +- SortMergeJoin Inner (736)
: : : :- Sort (644)
: : : : +- Exchange (643)
: : : : +- Project (642)
: : : : +- SortMergeJoin Inner (641)
: : : : :- Sort (612)
: : : : : +- Exchange (611)
: : : : : +- Project (610)
: : : : : +- Filter (609)
: : : : : +- Generate (608)
: : : : : +- Filter (607)
: : : : : +- ObjectHashAggregate (606)
: : : : : +- ObjectHashAggregate (605)
: : : : : +- Generate (604)
: : : : : +- BroadcastHashJoin Inner BuildLeft (603)
: : : : : :- BroadcastExchange (560)
: : : : : : +- Exchange (559)
: : : : : : +- Project (558)
: : : : : : +- Filter (557)
: : : : : : +- Scan parquet (556)
: : : : : +- Project (602)
: : : : : +- BroadcastHashJoin Inner BuildLeft (601)
: : : : : :- BroadcastExchange (570)
: : : : : : +- Project (569)
: : : : : : +- Generate (568)
: : : : : : +- Project (567)
: : : : : : +- Filter (566)
: : : : : : +- Generate (565)
: : : : : : +- Exchange (564)
: : : : : : +- Project (563)
: : : : : : +- Filter (562)
: : : : : : +- Scan parquet (561)
: : : : : +- HashAggregate (600)
: : : : : +- Exchange (599)
: : : : : +- HashAggregate (598)
: : : : : +- Project (597)
: : : : : +- Generate (596)
: : : : : +- Project (595)
: : : : : +- Generate (594)
: : : : : +- Project (593)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (592)
: : : : : :- Project (586)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (585)
: : : : : : :- Project (580)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (579)
: : : : : : : :- Exchange (573)
: : : : : : : : +- Filter (572)
: : : : : : : : +- Scan parquet (571)
: : : : : : : +- BroadcastExchange (578)
: : : : : : : +- Project (577)
: : : : : : : +- Exchange (576)
: : : : : : : +- Project (575)
: : : : : : : +- Scan parquet (574)
: : : : : : +- BroadcastExchange (584)
: : : : : : +- Exchange (583)
: : : : : : +- Project (582)
: : : : : : +- Scan parquet (581)
: : : : : +- BroadcastExchange (591)
: : : : : +- Exchange (590)
: : : : : +- Project (589)
: : : : : +- Filter (588)
: : : : : +- Scan parquet (587)
: : : : +- Sort (640)
: : : : +- Project (639)
: : : : +- Generate (638)
: : : : +- Project (637)
: : : : +- Generate (636)
: : : : +- Project (635)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (634)
: : : : :- Project (628)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (627)
: : : : : :- Project (622)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (621)
: : : : : : :- Exchange (615)
: : : : : : : +- Filter (614)
: : : : : : : +- Scan parquet (613)
: : : : : : +- BroadcastExchange (620)
: : : : : : +- Project (619)
: : : : : : +- Exchange (618)
: : : : : : +- Project (617)
: : : : : : +- Scan parquet (616)
: : : : : +- BroadcastExchange (626)
: : : : : +- Exchange (625)
: : : : : +- Project (624)
: : : : : +- Scan parquet (623)
: : : : +- BroadcastExchange (633)
: : : : +- Exchange (632)
: : : : +- Project (631)
: : : : +- Filter (630)
: : : : +- Scan parquet (629)
: : : +- Sort (735)
: : : +- Exchange (734)
: : : +- HashAggregate (733)
: : : +- HashAggregate (732)
: : : +- Project (731)
: : : +- SortMergeJoin Inner (730)
: : : :- Sort (701)
: : : : +- Exchange (700)
: : : : +- Project (699)
: : : : +- Filter (698)
: : : : +- Generate (697)
: : : : +- Filter (696)
: : : : +- ObjectHashAggregate (695)
: : : : +- ObjectHashAggregate (694)
: : : : +- Generate (693)
: : : : +- BroadcastHashJoin Inner BuildLeft (692)
: : : : :- BroadcastExchange (649)
: : : : : +- Exchange (648)
: : : : : +- Project (647)
: : : : : +- Filter (646)
: : : : : +- Scan parquet (645)
: : : : +- Project (691)
: : : : +- BroadcastHashJoin Inner BuildLeft (690)
: : : : :- BroadcastExchange (659)
: : : : : +- Project (658)
: : : : : +- Generate (657)
: : : : : +- Project (656)
: : : : : +- Filter (655)
: : : : : +- Generate (654)
: : : : : +- Exchange (653)
: : : : : +- Project (652)
: : : : : +- Filter (651)
: : : : : +- Scan parquet (650)
: : : : +- HashAggregate (689)
: : : : +- Exchange (688)
: : : : +- HashAggregate (687)
: : : : +- Project (686)
: : : : +- Generate (685)
: : : : +- Project (684)
: : : : +- Generate (683)
: : : : +- Project (682)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (681)
: : : : :- Project (675)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (674)
: : : : : :- Project (669)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (668)
: : : : : : :- Exchange (662)
: : : : : : : +- Filter (661)
: : : : : : : +- Scan parquet (660)
: : : : : : +- BroadcastExchange (667)
: : : : : : +- Project (666)
: : : : : : +- Exchange (665)
: : : : : : +- Project (664)
: : : : : : +- Scan parquet (663)
: : : : : +- BroadcastExchange (673)
: : : : : +- Exchange (672)
: : : : : +- Project (671)
: : : : : +- Scan parquet (670)
: : : : +- BroadcastExchange (680)
: : : : +- Exchange (679)
: : : : +- Project (678)
: : : : +- Filter (677)
: : : : +- Scan parquet (676)
: : : +- Sort (729)
: : : +- Project (728)
: : : +- Generate (727)
: : : +- Project (726)
: : : +- Generate (725)
: : : +- Project (724)
: : : +- BroadcastHashJoin LeftOuter BuildRight (723)
: : : :- Project (717)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (716)
: : : : :- Project (711)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (710)
: : : : : :- Exchange (704)
: : : : : : +- Filter (703)
: : : : : : +- Scan parquet (702)
: : : : : +- BroadcastExchange (709)
: : : : : +- Project (708)
: : : : : +- Exchange (707)
: : : : : +- Project (706)
: : : : : +- Scan parquet (705)
: : : : +- BroadcastExchange (715)
: : : : +- Exchange (714)
: : : : +- Project (713)
: : : : +- Scan parquet (712)
: : : +- BroadcastExchange (722)
: : : +- Exchange (721)
: : : +- Project (720)
: : : +- Filter (719)
: : : +- Scan parquet (718)
: : +- BroadcastExchange (745)
: : +- Filter (744)
: : +- Scan parquet (267)
: +- Sort (802)
: +- Project (801)
: +- BroadcastHashJoin Inner BuildLeft (800)
: :- BroadcastExchange (769)
: : +- Project (768)
: : +- Filter (767)
: : +- Generate (766)
: : +- Project (765)
: : +- Filter (764)
: : +- Generate (763)
: : +- Exchange (762)
: : +- Project (761)
: : +- Filter (760)
: : +- Scan parquet (759)
: +- HashAggregate (799)
: +- Exchange (798)
: +- HashAggregate (797)
: +- Project (796)
: +- Generate (795)
: +- Project (794)
: +- Generate (793)
: +- Project (792)
: +- BroadcastHashJoin LeftOuter BuildRight (791)
: :- Project (785)
: : +- BroadcastHashJoin LeftOuter BuildRight (784)
: : :- Project (779)
: : : +- BroadcastHashJoin LeftOuter BuildRight (778)
: : : :- Exchange (772)
: : : : +- Filter (771)
: : : : +- Scan parquet (770)
: : : +- BroadcastExchange (777)
: : : +- Project (776)
: : : +- Exchange (775)
: : : +- Project (774)
: : : +- Scan parquet (773)
: : +- BroadcastExchange (783)
: : +- Exchange (782)
: : +- Project (781)
: : +- Scan parquet (780)
: +- BroadcastExchange (790)
: +- Exchange (789)
: +- Project (788)
: +- Filter (787)
: +- Scan parquet (786)
+- BroadcastExchange (811)
+- Project (810)
+- Filter (809)
+- Generate (808)
+- Exchange (807)
+- Project (806)
+- Filter (805)
+- Scan parquet (308)
+- == Initial Plan ==
Filter (837)
+- ObjectHashAggregate (836)
+- Exchange (835)
+- ObjectHashAggregate (834)
+- Project (833)
+- Window (832)
+- Sort (831)
+- Exchange (830)
+- Project (829)
+- InMemoryTableScan (3)
+- InMemoryRelation (4)
+- AdaptiveSparkPlan (814)
+- == Final Plan ==
ResultQueryStage (321)
+- Project (320)
+- * BroadcastHashJoin Inner BuildRight (319)
:- * Project (307)
: +- * BroadcastHashJoin Inner BuildLeft (306)
: :- BroadcastQueryStage (290), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (289)
: : +- AQEShuffleRead (288)
: : +- ShuffleQueryStage (287), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : +- Exchange (286)
: : +- * Project (285)
: : +- * BroadcastHashJoin LeftOuter BuildRight (284)
: : :- AQEShuffleRead (178)
: : : +- ShuffleQueryStage (177), Statistics(sizeInBytes=114.8 KiB, rowCount=980)
: : : +- Exchange (176)
: : : +- * Project (175)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (174)
: : : :- * Project (42)
: : : : +- * Generate (41)
: : : : +- * Project (40)
: : : : +- * Generate (39)
: : : : +- Project (38)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (37)
: : : : :- * Project (28)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (27)
: : : : : :- * Project (19)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (18)
: : : : : : :- ShuffleQueryStage (9), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (8)
: : : : : : : +- * Filter (7)
: : : : : : : +- * ColumnarToRow (6)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastQueryStage (17), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (16)
: : : : : : +- * Project (15)
: : : : : : +- ShuffleQueryStage (14), Statistics(sizeInBytes=339.5 KiB, rowCount=2.75E+3)
: : : : : : +- Exchange (13)
: : : : : : +- * Project (12)
: : : : : : +- * ColumnarToRow (11)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastQueryStage (26), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- BroadcastExchange (25)
: : : : : +- ShuffleQueryStage (24), Statistics(sizeInBytes=4.4 KiB, rowCount=62)
: : : : : +- Exchange (23)
: : : : : +- * Project (22)
: : : : : +- * ColumnarToRow (21)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastQueryStage (36), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- BroadcastExchange (35)
: : : : +- ShuffleQueryStage (34), Statistics(sizeInBytes=92.7 KiB, rowCount=159)
: : : : +- Exchange (33)
: : : : +- * Project (32)
: : : : +- * Filter (31)
: : : : +- * ColumnarToRow (30)
: : : : +- Scan parquet (29)
: : : +- BroadcastQueryStage (173), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (172)
: : : +- * HashAggregate (171)
: : : +- AQEShuffleRead (170)
: : : +- ShuffleQueryStage (169), Statistics(sizeInBytes=27.7 KiB, rowCount=295)
: : : +- Exchange (168)
: : : +- * HashAggregate (167)
: : : +- * Project (166)
: : : +- * BroadcastHashJoin Inner BuildLeft (165)
: : : :- BroadcastQueryStage (137), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (136)
: : : : +- AQEShuffleRead (135)
: : : : +- ShuffleQueryStage (134), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : : +- Exchange (133)
: : : : +- * Project (132)
: : : : +- * BroadcastHashJoin Inner BuildLeft (131)
: : : : :- BroadcastQueryStage (109), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (108)
: : : : : +- AQEShuffleRead (107)
: : : : : +- ShuffleQueryStage (106), Statistics(sizeInBytes=43.1 KiB, rowCount=394)
: : : : : +- Exchange (105)
: : : : : +- * Project (104)
: : : : : +- * Filter (103)
: : : : : +- * Generate (102)
: : : : : +- * Filter (101)
: : : : : +- ObjectHashAggregate (100)
: : : : : +- ObjectHashAggregate (99)
: : : : : +- Generate (98)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (97)
: : : : : :- BroadcastQueryStage (50), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- BroadcastExchange (49)
: : : : : : +- ShuffleQueryStage (48), Statistics(sizeInBytes=240.1 KiB, rowCount=398)
: : : : : : +- Exchange (47)
: : : : : : +- * Project (46)
: : : : : : +- * Filter (45)
: : : : : : +- * ColumnarToRow (44)
: : : : : : +- Scan parquet (43)
: : : : : +- * Project (96)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (95)
: : : : : :- BroadcastQueryStage (63), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- BroadcastExchange (62)
: : : : : : +- * Project (61)
: : : : : : +- * Generate (60)
: : : : : : +- * Project (59)
: : : : : : +- * Filter (58)
: : : : : : +- * Generate (57)
: : : : : : +- ShuffleQueryStage (56), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : : : : : +- Exchange (55)
: : : : : : +- * Project (54)
: : : : : : +- * Filter (53)
: : : : : : +- * ColumnarToRow (52)
: : : : : : +- Scan parquet (51)
: : : : : +- * HashAggregate (94)
: : : : : +- AQEShuffleRead (93)
: : : : : +- ShuffleQueryStage (92), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- Exchange (91)
: : : : : +- * HashAggregate (90)
: : : : : +- * Project (89)
: : : : : +- * Generate (88)
: : : : : +- * Project (87)
: : : : : +- * Generate (86)
: : : : : +- Project (85)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (84)
: : : : : :- * Project (81)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (80)
: : : : : : :- * Project (72)
: : : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (71)
: : : : : : : :- ShuffleQueryStage (68), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : : : : : : : +- Exchange (67)
: : : : : : : : +- * Filter (66)
: : : : : : : : +- * ColumnarToRow (65)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastQueryStage (70), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : : +- ReusedExchange (69)
: : : : : : +- BroadcastQueryStage (79), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : : +- BroadcastExchange (78)
: : : : : : +- ShuffleQueryStage (77), Statistics(sizeInBytes=2.9 KiB, rowCount=62)
: : : : : : +- Exchange (76)
: : : : : : +- * Project (75)
: : : : : : +- * ColumnarToRow (74)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastQueryStage (83), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : : +- ReusedExchange (82)
: : : : +- * Project (130)
: : : : +- * Generate (129)
: : : : +- * Project (128)
: : : : +- * Generate (127)
: : : : +- Project (126)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (125)
: : : : :- * Project (122)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (121)
: : : : : :- * Project (118)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (117)
: : : : : : :- ShuffleQueryStage (114), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (113)
: : : : : : : +- * Filter (112)
: : : : : : : +- * ColumnarToRow (111)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastQueryStage (116), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (115)
: : : : : +- BroadcastQueryStage (120), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (119)
: : : : +- BroadcastQueryStage (124), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (123)
: : : +- AQEShuffleRead (164)
: : : +- ShuffleQueryStage (163), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- Exchange (162)
: : : +- * HashAggregate (161)
: : : +- * HashAggregate (160)
: : : +- * Project (159)
: : : +- * BroadcastHashJoin Inner BuildLeft (158)
: : : :- BroadcastQueryStage (139), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : +- ReusedExchange (138)
: : : +- * Project (157)
: : : +- * Generate (156)
: : : +- * Project (155)
: : : +- * Generate (154)
: : : +- Project (153)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (152)
: : : :- * Project (149)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (148)
: : : : :- * Project (145)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (144)
: : : : : :- ShuffleQueryStage (141), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : +- ReusedExchange (140)
: : : : : +- BroadcastQueryStage (143), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : +- ReusedExchange (142)
: : : : +- BroadcastQueryStage (147), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : +- ReusedExchange (146)
: : : +- BroadcastQueryStage (151), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : +- ReusedExchange (150)
: : +- BroadcastQueryStage (283), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (282)
: : +- AQEShuffleRead (281)
: : +- ShuffleQueryStage (280), Statistics(sizeInBytes=84.2 KiB, rowCount=980)
: : +- Exchange (279)
: : +- * SerializeFromObject (278)
: : +- * MapElements (277)
: : +- * DeserializeToObject (276)
: : +- ObjectHashAggregate (275)
: : +- ObjectHashAggregate (274)
: : +- * Project (273)
: : +- * BroadcastHashJoin LeftOuter BuildRight (272)
: : :- * Project (266)
: : : +- * BroadcastHashJoin LeftOuter BuildRight (265)
: : : :- * Project (204)
: : : : +- * Project (203)
: : : : +- * Generate (202)
: : : : +- * Generate (201)
: : : : +- * Project (200)
: : : : +- Project (199)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (198)
: : : : :- * Project (195)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (194)
: : : : : :- * BroadcastHashJoin LeftOuter BuildRight (191)
: : : : : : :- ShuffleQueryStage (182), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- Exchange (181)
: : : : : : : +- * ColumnarToRow (180)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastQueryStage (190), Statistics(sizeInBytes=4.1 MiB, rowCount=2.75E+3)
: : : : : : +- BroadcastExchange (189)
: : : : : : +- * Project (188)
: : : : : : +- ShuffleQueryStage (187), Statistics(sizeInBytes=2.0 MiB, rowCount=2.75E+3)
: : : : : : +- Exchange (186)
: : : : : : +- * Project (185)
: : : : : : +- * ColumnarToRow (184)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastQueryStage (193), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (192)
: : : : +- BroadcastQueryStage (197), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (196)
: : : +- BroadcastQueryStage (264), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : +- BroadcastExchange (263)
: : : +- * HashAggregate (262)
: : : +- AQEShuffleRead (261)
: : : +- ShuffleQueryStage (260), Statistics(sizeInBytes=30.0 KiB, rowCount=295)
: : : +- Exchange (259)
: : : +- * HashAggregate (258)
: : : +- * Project (257)
: : : +- * BroadcastHashJoin Inner BuildLeft (256)
: : : :- BroadcastQueryStage (252), Statistics(sizeInBytes=2.0 MiB, rowCount=295)
: : : : +- BroadcastExchange (251)
: : : : +- AQEShuffleRead (250)
: : : : +- ShuffleQueryStage (249), Statistics(sizeInBytes=34.6 KiB, rowCount=295)
: : : : +- Exchange (248)
: : : : +- * Project (247)
: : : : +- * BroadcastHashJoin Inner BuildLeft (246)
: : : : :- BroadcastQueryStage (227), Statistics(sizeInBytes=2.0 MiB, rowCount=394)
: : : : : +- BroadcastExchange (226)
: : : : : +- AQEShuffleRead (225)
: : : : : +- ShuffleQueryStage (224), Statistics(sizeInBytes=46.2 KiB, rowCount=394)
: : : : : +- Exchange (223)
: : : : : +- * Project (222)
: : : : : +- * Filter (221)
: : : : : +- * Generate (220)
: : : : : +- * Filter (219)
: : : : : +- ObjectHashAggregate (218)
: : : : : +- ObjectHashAggregate (217)
: : : : : +- Generate (216)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (215)
: : : : : :- BroadcastQueryStage (206), Statistics(sizeInBytes=2.0 MiB, rowCount=398)
: : : : : : +- ReusedExchange (205)
: : : : : +- * Project (214)
: : : : : +- * BroadcastHashJoin Inner BuildLeft (213)
: : : : : :- BroadcastQueryStage (208), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : : : : : +- ReusedExchange (207)
: : : : : +- * HashAggregate (212)
: : : : : +- AQEShuffleRead (211)
: : : : : +- ShuffleQueryStage (210), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: : : : : +- ReusedExchange (209)
: : : : +- * Project (245)
: : : : +- * Generate (244)
: : : : +- * Project (243)
: : : : +- * Generate (242)
: : : : +- Project (241)
: : : : +- * BroadcastHashJoin LeftOuter BuildRight (240)
: : : : :- * Project (237)
: : : : : +- * BroadcastHashJoin LeftOuter BuildRight (236)
: : : : : :- * Project (233)
: : : : : : +- * BroadcastHashJoin LeftOuter BuildRight (232)
: : : : : : :- ShuffleQueryStage (229), Statistics(sizeInBytes=91.9 KiB, rowCount=980)
: : : : : : : +- ReusedExchange (228)
: : : : : : +- BroadcastQueryStage (231), Statistics(sizeInBytes=2.1 MiB, rowCount=2.75E+3)
: : : : : : +- ReusedExchange (230)
: : : : : +- BroadcastQueryStage (235), Statistics(sizeInBytes=2.0 MiB, rowCount=62)
: : : : : +- ReusedExchange (234)
: : : : +- BroadcastQueryStage (239), Statistics(sizeInBytes=2.0 MiB, rowCount=159)
: : : : +- ReusedExchange (238)
: : : +- AQEShuffleRead (255)
: : : +- ShuffleQueryStage (254), Statistics(sizeInBytes=32.3 KiB, rowCount=295)
: : : +- ReusedExchange (253)
: : +- BroadcastQueryStage (271), Statistics(sizeInBytes=2.0 MiB, rowCount=980)
: : +- BroadcastExchange (270)
: : +- * Filter (269)
: : +- * ColumnarToRow (268)
: : +- Scan parquet (267)
: +- * Project (305)
: +- * BroadcastHashJoin Inner BuildLeft (304)
: :- BroadcastQueryStage (300), Statistics(sizeInBytes=2.0 MiB, rowCount=11)
: : +- BroadcastExchange (299)
: : +- * Project (298)
: : +- * Filter (297)
: : +- * Generate (296)
: : +- * Project (295)
: : +- * Filter (294)
: : +- * Generate (293)
: : +- ShuffleQueryStage (292), Statistics(sizeInBytes=1880.0 B, rowCount=1)
: : +- ReusedExchange (291)
: +- * HashAggregate (303)
: +- ShuffleQueryStage (302), Statistics(sizeInBytes=11.5 KiB, rowCount=245)
: +- ReusedExchange (301)
+- BroadcastQueryStage (318), Statistics(sizeInBytes=2.0 MiB, rowCount=44)
+- BroadcastExchange (317)
+- * Project (316)
+- * Filter (315)
+- * Generate (314)
+- ShuffleQueryStage (313), Statistics(sizeInBytes=29.9 KiB, rowCount=9)
+- Exchange (312)
+- * Project (311)
+- * Filter (310)
+- * ColumnarToRow (309)
+- Scan parquet (308)
+- == Initial Plan ==
Project (813)
+- BroadcastHashJoin Inner BuildRight (812)
:- Project (804)
: +- SortMergeJoin Inner (803)
: :- Sort (758)
: : +- Exchange (757)
: : +- Project (756)
: : +- SortMergeJoin LeftOuter (755)
: : :- Sort (530)
: : : +- Exchange (529)
: : : +- Project (528)
: : : +- SortMergeJoin LeftOuter (527)
: : : :- Sort (345)
: : : : +- Project (344)
: : : : +- Generate (343)
: : : : +- Project (342)
: : : : +- Generate (341)
: : : : +- Project (340)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (339)
: : : : :- Project (334)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (333)
: : : : : :- Project (329)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (328)
: : : : : : :- Exchange (323)
: : : : : : : +- Filter (322)
: : : : : : : +- Scan parquet (5)
: : : : : : +- BroadcastExchange (327)
: : : : : : +- Project (326)
: : : : : : +- Exchange (325)
: : : : : : +- Project (324)
: : : : : : +- Scan parquet (10)
: : : : : +- BroadcastExchange (332)
: : : : : +- Exchange (331)
: : : : : +- Project (330)
: : : : : +- Scan parquet (20)
: : : : +- BroadcastExchange (338)
: : : : +- Exchange (337)
: : : : +- Project (336)
: : : : +- Filter (335)
: : : : +- Scan parquet (29)
: : : +- Sort (526)
: : : +- HashAggregate (525)
: : : +- Exchange (524)
: : : +- HashAggregate (523)
: : : +- Project (522)
: : : +- SortMergeJoin Inner (521)
: : : :- Sort (429)
: : : : +- Exchange (428)
: : : : +- Project (427)
: : : : +- SortMergeJoin Inner (426)
: : : : :- Sort (398)
: : : : : +- Exchange (397)
: : : : : +- Project (396)
: : : : : +- Filter (395)
: : : : : +- Generate (394)
: : : : : +- Filter (393)
: : : : : +- ObjectHashAggregate (392)
: : : : : +- ObjectHashAggregate (391)
: : : : : +- Generate (390)
: : : : : +- BroadcastHashJoin Inner BuildLeft (389)
: : : : : :- BroadcastExchange (349)
: : : : : : +- Exchange (348)
: : : : : : +- Project (347)
: : : : : : +- Filter (346)
: : : : : : +- Scan parquet (43)
: : : : : +- Project (388)
: : : : : +- BroadcastHashJoin Inner BuildLeft (387)
: : : : : :- BroadcastExchange (358)
: : : : : : +- Project (357)
: : : : : : +- Generate (356)
: : : : : : +- Project (355)
: : : : : : +- Filter (354)
: : : : : : +- Generate (353)
: : : : : : +- Exchange (352)
: : : : : : +- Project (351)
: : : : : : +- Filter (350)
: : : : : : +- Scan parquet (51)
: : : : : +- HashAggregate (386)
: : : : : +- Exchange (385)
: : : : : +- HashAggregate (384)
: : : : : +- Project (383)
: : : : : +- Generate (382)
: : : : : +- Project (381)
: : : : : +- Generate (380)
: : : : : +- Project (379)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (378)
: : : : : :- Project (372)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (371)
: : : : : : :- Project (367)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (366)
: : : : : : : :- Exchange (360)
: : : : : : : : +- Filter (359)
: : : : : : : : +- Scan parquet (64)
: : : : : : : +- BroadcastExchange (365)
: : : : : : : +- Project (364)
: : : : : : : +- Exchange (363)
: : : : : : : +- Project (362)
: : : : : : : +- Scan parquet (361)
: : : : : : +- BroadcastExchange (370)
: : : : : : +- Exchange (369)
: : : : : : +- Project (368)
: : : : : : +- Scan parquet (73)
: : : : : +- BroadcastExchange (377)
: : : : : +- Exchange (376)
: : : : : +- Project (375)
: : : : : +- Filter (374)
: : : : : +- Scan parquet (373)
: : : : +- Sort (425)
: : : : +- Project (424)
: : : : +- Generate (423)
: : : : +- Project (422)
: : : : +- Generate (421)
: : : : +- Project (420)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (419)
: : : : :- Project (413)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (412)
: : : : : :- Project (407)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (406)
: : : : : : :- Exchange (400)
: : : : : : : +- Filter (399)
: : : : : : : +- Scan parquet (110)
: : : : : : +- BroadcastExchange (405)
: : : : : : +- Project (404)
: : : : : : +- Exchange (403)
: : : : : : +- Project (402)
: : : : : : +- Scan parquet (401)
: : : : : +- BroadcastExchange (411)
: : : : : +- Exchange (410)
: : : : : +- Project (409)
: : : : : +- Scan parquet (408)
: : : : +- BroadcastExchange (418)
: : : : +- Exchange (417)
: : : : +- Project (416)
: : : : +- Filter (415)
: : : : +- Scan parquet (414)
: : : +- Sort (520)
: : : +- Exchange (519)
: : : +- HashAggregate (518)
: : : +- HashAggregate (517)
: : : +- Project (516)
: : : +- SortMergeJoin Inner (515)
: : : :- Sort (486)
: : : : +- Exchange (485)
: : : : +- Project (484)
: : : : +- Filter (483)
: : : : +- Generate (482)
: : : : +- Filter (481)
: : : : +- ObjectHashAggregate (480)
: : : : +- ObjectHashAggregate (479)
: : : : +- Generate (478)
: : : : +- BroadcastHashJoin Inner BuildLeft (477)
: : : : :- BroadcastExchange (434)
: : : : : +- Exchange (433)
: : : : : +- Project (432)
: : : : : +- Filter (431)
: : : : : +- Scan parquet (430)
: : : : +- Project (476)
: : : : +- BroadcastHashJoin Inner BuildLeft (475)
: : : : :- BroadcastExchange (444)
: : : : : +- Project (443)
: : : : : +- Generate (442)
: : : : : +- Project (441)
: : : : : +- Filter (440)
: : : : : +- Generate (439)
: : : : : +- Exchange (438)
: : : : : +- Project (437)
: : : : : +- Filter (436)
: : : : : +- Scan parquet (435)
: : : : +- HashAggregate (474)
: : : : +- Exchange (473)
: : : : +- HashAggregate (472)
: : : : +- Project (471)
: : : : +- Generate (470)
: : : : +- Project (469)
: : : : +- Generate (468)
: : : : +- Project (467)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (466)
: : : : :- Project (460)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (459)
: : : : : :- Project (454)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (453)
: : : : : : :- Exchange (447)
: : : : : : : +- Filter (446)
: : : : : : : +- Scan parquet (445)
: : : : : : +- BroadcastExchange (452)
: : : : : : +- Project (451)
: : : : : : +- Exchange (450)
: : : : : : +- Project (449)
: : : : : : +- Scan parquet (448)
: : : : : +- BroadcastExchange (458)
: : : : : +- Exchange (457)
: : : : : +- Project (456)
: : : : : +- Scan parquet (455)
: : : : +- BroadcastExchange (465)
: : : : +- Exchange (464)
: : : : +- Project (463)
: : : : +- Filter (462)
: : : : +- Scan parquet (461)
: : : +- Sort (514)
: : : +- Project (513)
: : : +- Generate (512)
: : : +- Project (511)
: : : +- Generate (510)
: : : +- Project (509)
: : : +- BroadcastHashJoin LeftOuter BuildRight (508)
: : : :- Project (502)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (501)
: : : : :- Project (496)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (495)
: : : : : :- Exchange (489)
: : : : : : +- Filter (488)
: : : : : : +- Scan parquet (487)
: : : : : +- BroadcastExchange (494)
: : : : : +- Project (493)
: : : : : +- Exchange (492)
: : : : : +- Project (491)
: : : : : +- Scan parquet (490)
: : : : +- BroadcastExchange (500)
: : : : +- Exchange (499)
: : : : +- Project (498)
: : : : +- Scan parquet (497)
: : : +- BroadcastExchange (507)
: : : +- Exchange (506)
: : : +- Project (505)
: : : +- Filter (504)
: : : +- Scan parquet (503)
: : +- Sort (754)
: : +- Exchange (753)
: : +- SerializeFromObject (752)
: : +- MapElements (751)
: : +- DeserializeToObject (750)
: : +- ObjectHashAggregate (749)
: : +- ObjectHashAggregate (748)
: : +- Project (747)
: : +- BroadcastHashJoin LeftOuter BuildRight (746)
: : :- Project (743)
: : : +- SortMergeJoin LeftOuter (742)
: : : :- Sort (555)
: : : : +- Project (554)
: : : : +- Project (553)
: : : : +- Generate (552)
: : : : +- Generate (551)
: : : : +- Project (550)
: : : : +- Project (549)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (548)
: : : : :- Project (542)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (541)
: : : : : :- BroadcastHashJoin LeftOuter BuildRight (536)
: : : : : : :- Exchange (531)
: : : : : : : +- Scan parquet (179)
: : : : : : +- BroadcastExchange (535)
: : : : : : +- Project (534)
: : : : : : +- Exchange (533)
: : : : : : +- Project (532)
: : : : : : +- Scan parquet (183)
: : : : : +- BroadcastExchange (540)
: : : : : +- Exchange (539)
: : : : : +- Project (538)
: : : : : +- Scan parquet (537)
: : : : +- BroadcastExchange (547)
: : : : +- Exchange (546)
: : : : +- Project (545)
: : : : +- Filter (544)
: : : : +- Scan parquet (543)
: : : +- Sort (741)
: : : +- HashAggregate (740)
: : : +- Exchange (739)
: : : +- HashAggregate (738)
: : : +- Project (737)
: : : +- SortMergeJoin Inner (736)
: : : :- Sort (644)
: : : : +- Exchange (643)
: : : : +- Project (642)
: : : : +- SortMergeJoin Inner (641)
: : : : :- Sort (612)
: : : : : +- Exchange (611)
: : : : : +- Project (610)
: : : : : +- Filter (609)
: : : : : +- Generate (608)
: : : : : +- Filter (607)
: : : : : +- ObjectHashAggregate (606)
: : : : : +- ObjectHashAggregate (605)
: : : : : +- Generate (604)
: : : : : +- BroadcastHashJoin Inner BuildLeft (603)
: : : : : :- BroadcastExchange (560)
: : : : : : +- Exchange (559)
: : : : : : +- Project (558)
: : : : : : +- Filter (557)
: : : : : : +- Scan parquet (556)
: : : : : +- Project (602)
: : : : : +- BroadcastHashJoin Inner BuildLeft (601)
: : : : : :- BroadcastExchange (570)
: : : : : : +- Project (569)
: : : : : : +- Generate (568)
: : : : : : +- Project (567)
: : : : : : +- Filter (566)
: : : : : : +- Generate (565)
: : : : : : +- Exchange (564)
: : : : : : +- Project (563)
: : : : : : +- Filter (562)
: : : : : : +- Scan parquet (561)
: : : : : +- HashAggregate (600)
: : : : : +- Exchange (599)
: : : : : +- HashAggregate (598)
: : : : : +- Project (597)
: : : : : +- Generate (596)
: : : : : +- Project (595)
: : : : : +- Generate (594)
: : : : : +- Project (593)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (592)
: : : : : :- Project (586)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (585)
: : : : : : :- Project (580)
: : : : : : : +- BroadcastHashJoin LeftOuter BuildRight (579)
: : : : : : : :- Exchange (573)
: : : : : : : : +- Filter (572)
: : : : : : : : +- Scan parquet (571)
: : : : : : : +- BroadcastExchange (578)
: : : : : : : +- Project (577)
: : : : : : : +- Exchange (576)
: : : : : : : +- Project (575)
: : : : : : : +- Scan parquet (574)
: : : : : : +- BroadcastExchange (584)
: : : : : : +- Exchange (583)
: : : : : : +- Project (582)
: : : : : : +- Scan parquet (581)
: : : : : +- BroadcastExchange (591)
: : : : : +- Exchange (590)
: : : : : +- Project (589)
: : : : : +- Filter (588)
: : : : : +- Scan parquet (587)
: : : : +- Sort (640)
: : : : +- Project (639)
: : : : +- Generate (638)
: : : : +- Project (637)
: : : : +- Generate (636)
: : : : +- Project (635)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (634)
: : : : :- Project (628)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (627)
: : : : : :- Project (622)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (621)
: : : : : : :- Exchange (615)
: : : : : : : +- Filter (614)
: : : : : : : +- Scan parquet (613)
: : : : : : +- BroadcastExchange (620)
: : : : : : +- Project (619)
: : : : : : +- Exchange (618)
: : : : : : +- Project (617)
: : : : : : +- Scan parquet (616)
: : : : : +- BroadcastExchange (626)
: : : : : +- Exchange (625)
: : : : : +- Project (624)
: : : : : +- Scan parquet (623)
: : : : +- BroadcastExchange (633)
: : : : +- Exchange (632)
: : : : +- Project (631)
: : : : +- Filter (630)
: : : : +- Scan parquet (629)
: : : +- Sort (735)
: : : +- Exchange (734)
: : : +- HashAggregate (733)
: : : +- HashAggregate (732)
: : : +- Project (731)
: : : +- SortMergeJoin Inner (730)
: : : :- Sort (701)
: : : : +- Exchange (700)
: : : : +- Project (699)
: : : : +- Filter (698)
: : : : +- Generate (697)
: : : : +- Filter (696)
: : : : +- ObjectHashAggregate (695)
: : : : +- ObjectHashAggregate (694)
: : : : +- Generate (693)
: : : : +- BroadcastHashJoin Inner BuildLeft (692)
: : : : :- BroadcastExchange (649)
: : : : : +- Exchange (648)
: : : : : +- Project (647)
: : : : : +- Filter (646)
: : : : : +- Scan parquet (645)
: : : : +- Project (691)
: : : : +- BroadcastHashJoin Inner BuildLeft (690)
: : : : :- BroadcastExchange (659)
: : : : : +- Project (658)
: : : : : +- Generate (657)
: : : : : +- Project (656)
: : : : : +- Filter (655)
: : : : : +- Generate (654)
: : : : : +- Exchange (653)
: : : : : +- Project (652)
: : : : : +- Filter (651)
: : : : : +- Scan parquet (650)
: : : : +- HashAggregate (689)
: : : : +- Exchange (688)
: : : : +- HashAggregate (687)
: : : : +- Project (686)
: : : : +- Generate (685)
: : : : +- Project (684)
: : : : +- Generate (683)
: : : : +- Project (682)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (681)
: : : : :- Project (675)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (674)
: : : : : :- Project (669)
: : : : : : +- BroadcastHashJoin LeftOuter BuildRight (668)
: : : : : : :- Exchange (662)
: : : : : : : +- Filter (661)
: : : : : : : +- Scan parquet (660)
: : : : : : +- BroadcastExchange (667)
: : : : : : +- Project (666)
: : : : : : +- Exchange (665)
: : : : : : +- Project (664)
: : : : : : +- Scan parquet (663)
: : : : : +- BroadcastExchange (673)
: : : : : +- Exchange (672)
: : : : : +- Project (671)
: : : : : +- Scan parquet (670)
: : : : +- BroadcastExchange (680)
: : : : +- Exchange (679)
: : : : +- Project (678)
: : : : +- Filter (677)
: : : : +- Scan parquet (676)
: : : +- Sort (729)
: : : +- Project (728)
: : : +- Generate (727)
: : : +- Project (726)
: : : +- Generate (725)
: : : +- Project (724)
: : : +- BroadcastHashJoin LeftOuter BuildRight (723)
: : : :- Project (717)
: : : : +- BroadcastHashJoin LeftOuter BuildRight (716)
: : : : :- Project (711)
: : : : : +- BroadcastHashJoin LeftOuter BuildRight (710)
: : : : : :- Exchange (704)
: : : : : : +- Filter (703)
: : : : : : +- Scan parquet (702)
: : : : : +- BroadcastExchange (709)
: : : : : +- Project (708)
: : : : : +- Exchange (707)
: : : : : +- Project (706)
: : : : : +- Scan parquet (705)
: : : : +- BroadcastExchange (715)
: : : : +- Exchange (714)
: : : : +- Project (713)
: : : : +- Scan parquet (712)
: : : +- BroadcastExchange (722)
: : : +- Exchange (721)
: : : +- Project (720)
: : : +- Filter (719)
: : : +- Scan parquet (718)
: : +- BroadcastExchange (745)
: : +- Filter (744)
: : +- Scan parquet (267)
: +- Sort (802)
: +- Project (801)
: +- BroadcastHashJoin Inner BuildLeft (800)
: :- BroadcastExchange (769)
: : +- Project (768)
: : +- Filter (767)
: : +- Generate (766)
: : +- Project (765)
: : +- Filter (764)
: : +- Generate (763)
: : +- Exchange (762)
: : +- Project (761)
: : +- Filter (760)
: : +- Scan parquet (759)
: +- HashAggregate (799)
: +- Exchange (798)
: +- HashAggregate (797)
: +- Project (796)
: +- Generate (795)
: +- Project (794)
: +- Generate (793)
: +- Project (792)
: +- BroadcastHashJoin LeftOuter BuildRight (791)
: :- Project (785)
: : +- BroadcastHashJoin LeftOuter BuildRight (784)
: : :- Project (779)
: : : +- BroadcastHashJoin LeftOuter BuildRight (778)
: : : :- Exchange (772)
: : : : +- Filter (771)
: : : : +- Scan parquet (770)
: : : +- BroadcastExchange (777)
: : : +- Project (776)
: : : +- Exchange (775)
: : : +- Project (774)
: : : +- Scan parquet (773)
: : +- BroadcastExchange (783)
: : +- Exchange (782)
: : +- Project (781)
: : +- Scan parquet (780)
: +- BroadcastExchange (790)
: +- Exchange (789)
: +- Project (788)
: +- Filter (787)
: +- Scan parquet (786)
+- BroadcastExchange (811)
+- Project (810)
+- Filter (809)
+- Generate (808)
+- Exchange (807)
+- Project (806)
+- Filter (805)
+- Scan parquet (308)
(1) InMemoryTableScan
Output [3]: [openGoodsMovements#75050, plant#70770, product#70773]
Arguments: [openGoodsMovements#75050, plant#70770, product#70773], [(size(transform(openGoodsMovements#75050, lambdafunction(struct(documentType, PO, documentId, lambda x_27#90488.documentId, documentUuid, lambda x_27#90488.documentUuid, openQty, lambda x_27#90488.openQty, qtyUomUuid, lambda x_27#90488.qtyUomUuid, qtyUom, lambda x_27#90488.qtyUom, plannedDeliveryDatetime, lambda x_27#90488.availabilityDatetime), lambda x_27#90488, false)), false) > 0)]
(2) InMemoryRelation
Arguments: [product#70773, plant#70770, openGoodsMovements#75050], StorageLevel(disk, memory, 2 replicas)
(3) InMemoryTableScan
Output [10]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityDenominator#70782, quantityNumerator#70781, quantityUom#70779, quantityUomUuid#70780, scheduleLines#70783, validConfirmations#70963]
Arguments: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityDenominator#70782, quantityNumerator#70781, quantityUom#70779, quantityUomUuid#70780, scheduleLines#70783, validConfirmations#70963]
(4) InMemoryRelation
Arguments: [id#70731, internalUUID#70730, supplierUuid#70747, plant#70770, product#70773, externalPlantId#70775, externalProductId#70777, storageLocUuid#70778, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, confirmations#70784, stockDatetime#69868, timezonecode#69079, validConfirmations#70963], StorageLevel(disk, memory, deserialized, 1 replicas)
(5) Scan parquet
Output [3]: [product#69019, plant#69020, orderDateTime#69021]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
PushedFilters: [IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(6) ColumnarToRow [codegen id : 1]
Input [3]: [product#69019, plant#69020, orderDateTime#69021]
(7) Filter [codegen id : 1]
Input [3]: [product#69019, plant#69020, orderDateTime#69021]
Condition : (isnotnull(plant#69020) AND isnotnull(product#69019))
(8) Exchange
Input [3]: [product#69019, plant#69020, orderDateTime#69021]
Arguments: hashpartitioning(product#69019, plant#69020, 25), REPARTITION_BY_NUM, [plan_id=51871]
(9) ShuffleQueryStage
Output [3]: [product#69019, plant#69020, orderDateTime#69021]
Arguments: 0
(10) Scan parquet
Output [4]: [plant#69029, product#69031, productionAspect#69036, systemId#69025]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(11) ColumnarToRow [codegen id : 2]
Input [4]: [plant#69029, product#69031, productionAspect#69036, systemId#69025]
(12) Project [codegen id : 2]
Output [4]: [plant#69029.internalRefUUID AS _extract_internalRefUUID#71360, product#69031.internalRefUUID AS _extract_internalRefUUID#71357, productionAspect#69036.productMovementPlants.unitOfIssue.code AS _extract_code#71361, productionAspect#69036.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71362]
Input [4]: [plant#69029, product#69031, productionAspect#69036, systemId#69025]
(13) Exchange
Input [4]: [_extract_internalRefUUID#71360, _extract_internalRefUUID#71357, _extract_code#71361, _extract_internalRefUUID#71362]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=51887]
(14) ShuffleQueryStage
Output [4]: [_extract_internalRefUUID#71360, _extract_internalRefUUID#71357, _extract_code#71361, _extract_internalRefUUID#71362]
Arguments: 1
(15) Project [codegen id : 61]
Output [4]: [_extract_code#71361, _extract_internalRefUUID#71362, _extract_internalRefUUID#71357, _extract_internalRefUUID#71360]
Input [4]: [_extract_internalRefUUID#71360, _extract_internalRefUUID#71357, _extract_code#71361, _extract_internalRefUUID#71362]
(16) BroadcastExchange
Input [4]: [_extract_code#71361, _extract_internalRefUUID#71362, _extract_internalRefUUID#71357, _extract_internalRefUUID#71360]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=75325]
(17) BroadcastQueryStage
Output [4]: [_extract_code#71361, _extract_internalRefUUID#71362, _extract_internalRefUUID#71357, _extract_internalRefUUID#71360]
Arguments: 122
(18) BroadcastHashJoin [codegen id : 103]
Left keys [4]: [coalesce(product#69019, ), isnull(product#69019), coalesce(plant#69020, ), isnull(plant#69020)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71357, ), isnull(_extract_internalRefUUID#71357), coalesce(_extract_internalRefUUID#71360, ), isnull(_extract_internalRefUUID#71360)]
Join type: LeftOuter
Join condition: None
(19) Project [codegen id : 103]
Output [5]: [product#69019, plant#69020, orderDateTime#69021, _extract_code#71361, _extract_internalRefUUID#71362]
Input [7]: [product#69019, plant#69020, orderDateTime#69021, _extract_code#71361, _extract_internalRefUUID#71362, _extract_internalRefUUID#71357, _extract_internalRefUUID#71360]
(20) Scan parquet
Output [3]: [internalUUID#69062, address#69068, systemId#69061]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string,address:struct<timeZone:struct<code:string>>>
(21) ColumnarToRow [codegen id : 3]
Input [3]: [internalUUID#69062, address#69068, systemId#69061]
(22) Project [codegen id : 3]
Output [2]: [internalUUID#69062, address#69068.timeZone.code AS timezonecode#69074]
Input [3]: [internalUUID#69062, address#69068, systemId#69061]
(23) Exchange
Input [2]: [internalUUID#69062, timezonecode#69074]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=51906]
(24) ShuffleQueryStage
Output [2]: [internalUUID#69062, timezonecode#69074]
Arguments: 2
(25) BroadcastExchange
Input [2]: [internalUUID#69062, timezonecode#69074]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=66364]
(26) BroadcastQueryStage
Output [2]: [internalUUID#69062, timezonecode#69074]
Arguments: 105
(27) BroadcastHashJoin [codegen id : 103]
Left keys [2]: [coalesce(plant#69020, ), isnull(plant#69020)]
Right keys [2]: [coalesce(internalUUID#69062, ), isnull(internalUUID#69062)]
Join type: LeftOuter
Join condition: None
(28) Project [codegen id : 103]
Output [6]: [product#69019, plant#69020, orderDateTime#69021, _extract_code#71361, _extract_internalRefUUID#71362, CASE WHEN isnotnull(timezonecode#69074) THEN timezonecode#69074 ELSE UTC END AS timezonecode#69079]
Input [7]: [product#69019, plant#69020, orderDateTime#69021, _extract_code#71361, _extract_internalRefUUID#71362, internalUUID#69062, timezonecode#69074]
(29) Scan parquet
Output [4]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100, systemId#69093]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(30) ColumnarToRow [codegen id : 4]
Input [4]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100, systemId#69093]
(31) Filter [codegen id : 4]
Input [4]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100, systemId#69093]
Condition : isnotnull(internalUUID#69094)
(32) Project [codegen id : 4]
Output [3]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100]
Input [4]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100, systemId#69093]
(33) Exchange
Input [3]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=51926]
(34) ShuffleQueryStage
Output [3]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100]
Arguments: 3
(35) BroadcastExchange
Input [3]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=69575]
(36) BroadcastQueryStage
Output [3]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100]
Arguments: 107
(37) BroadcastHashJoin [codegen id : 103]
Left keys [1]: [product#69019]
Right keys [1]: [internalUUID#69094]
Join type: LeftOuter
Join condition: None
(38) Project
Output [5]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, filter(unitOfMeasureConversions#69100, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71361) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71362)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#69098.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69098.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#69107]
Input [9]: [product#69019, plant#69020, orderDateTime#69021, _extract_code#71361, _extract_internalRefUUID#71362, timezonecode#69079, internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100]
(39) Generate [codegen id : 104]
Input [5]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, filteredUnitOfMeasureConversions#69107]
Arguments: explode(filteredUnitOfMeasureConversions#69107.quantityNumerator), [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, filteredUnitOfMeasureConversions#69107], true, [quantityNumerator#69124]
(40) Project [codegen id : 104]
Output [5]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, filteredUnitOfMeasureConversions#69107.quantityDenominator AS _extract_quantityDenominator#70968]
Input [6]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, filteredUnitOfMeasureConversions#69107, quantityNumerator#69124]
(41) Generate [codegen id : 104]
Input [5]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, _extract_quantityDenominator#70968]
Arguments: explode(_extract_quantityDenominator#70968), [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079], true, [quantityDenominator#69129]
(42) Project [codegen id : 104]
Output [4]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079]
Input [5]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, quantityDenominator#69129]
(43) Scan parquet
Output [6]: [internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262, systemId#69257]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(44) ColumnarToRow [codegen id : 5]
Input [6]: [internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262, systemId#69257]
(45) Filter [codegen id : 5]
Input [6]: [internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262, systemId#69257]
Condition : (isnotnull(plant#69260.internalRefUUID) AND isnotnull(product#69259.internalRefUUID))
(46) Project [codegen id : 5]
Output [6]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262]
Input [6]: [internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262, systemId#69257]
(47) Exchange
Input [6]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=51953]
(48) ShuffleQueryStage
Output [6]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262]
Arguments: 4
(49) BroadcastExchange
Input [6]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=72940]
(50) BroadcastQueryStage
Output [6]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262]
Arguments: 118
(51) Scan parquet
Output [2]: [marketUnit#69135, distributionCenters#69141]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(52) ColumnarToRow [codegen id : 6]
Input [2]: [marketUnit#69135, distributionCenters#69141]
(53) Filter [codegen id : 6]
Input [2]: [marketUnit#69135, distributionCenters#69141]
Condition : (((marketUnit#69135 <=> Daily_grocery) AND (size(distributionCenters#69141, true) > 0)) AND isnotnull(distributionCenters#69141))
(54) Project [codegen id : 6]
Output [1]: [distributionCenters#69141]
Input [2]: [marketUnit#69135, distributionCenters#69141]
(55) Exchange
Input [1]: [distributionCenters#69141]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=51973]
(56) ShuffleQueryStage
Output [1]: [distributionCenters#69141]
Arguments: 5
(57) Generate [codegen id : 56]
Input [1]: [distributionCenters#69141]
Arguments: explode(distributionCenters#69141), false, [distributionCenters#69183]
(58) Filter [codegen id : 56]
Input [1]: [distributionCenters#69183]
Condition : (((size(distributionCenters#69183.storageLocations, true) > 0) AND isnotnull(distributionCenters#69183.storageLocations)) AND isnotnull(distributionCenters#69183.internalRefUUID))
(59) Project [codegen id : 56]
Output [2]: [distributionCenters#69183.internalRefUUID AS _extract_internalRefUUID#70977, distributionCenters#69183.storageLocations AS _extract_storageLocations#70978]
Input [1]: [distributionCenters#69183]
(60) Generate [codegen id : 56]
Input [2]: [_extract_internalRefUUID#70977, _extract_storageLocations#70978]
Arguments: explode(_extract_storageLocations#70978), [_extract_internalRefUUID#70977], false, [storageLocations#69197]
(61) Project [codegen id : 56]
Output [3]: [_extract_internalRefUUID#70977 AS plantUuid#69186, storageLocations#69197.id AS storageLocId#69199, storageLocations#69197.internalRefUUID AS storageLocUuid#69200]
Input [2]: [_extract_internalRefUUID#70977, storageLocations#69197]
(62) BroadcastExchange
Input [3]: [plantUuid#69186, storageLocId#69199, storageLocUuid#69200]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=65359]
(63) BroadcastQueryStage
Output [3]: [plantUuid#69186, storageLocId#69199, storageLocUuid#69200]
Arguments: 97
(64) Scan parquet
Output [2]: [product#69771, plant#69772]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(65) ColumnarToRow [codegen id : 7]
Input [2]: [product#69771, plant#69772]
(66) Filter [codegen id : 7]
Input [2]: [product#69771, plant#69772]
Condition : isnotnull(plant#69772)
(67) Exchange
Input [2]: [product#69771, plant#69772]
Arguments: hashpartitioning(product#69771, plant#69772, 25), REPARTITION_BY_NUM, [plan_id=51996]
(68) ShuffleQueryStage
Output [2]: [product#69771, plant#69772]
Arguments: 6
(69) ReusedExchange [Reuses operator id: 16]
Output [4]: [_extract_code#71369, _extract_internalRefUUID#71370, _extract_internalRefUUID#71367, _extract_internalRefUUID#71368]
(70) BroadcastQueryStage
Output [4]: [_extract_code#71369, _extract_internalRefUUID#71370, _extract_internalRefUUID#71367, _extract_internalRefUUID#71368]
Arguments: 124
(71) BroadcastHashJoin [codegen id : 73]
Left keys [4]: [coalesce(product#69771, ), isnull(product#69771), coalesce(plant#69772, ), isnull(plant#69772)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71367, ), isnull(_extract_internalRefUUID#71367), coalesce(_extract_internalRefUUID#71368, ), isnull(_extract_internalRefUUID#71368)]
Join type: LeftOuter
Join condition: None
(72) Project [codegen id : 73]
Output [4]: [product#69771, plant#69772, _extract_code#71369, _extract_internalRefUUID#71370]
Input [6]: [product#69771, plant#69772, _extract_code#71369, _extract_internalRefUUID#71370, _extract_internalRefUUID#71367, _extract_internalRefUUID#71368]
(73) Scan parquet
Output [2]: [internalUUID#69798, systemId#69797]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(74) ColumnarToRow [codegen id : 9]
Input [2]: [internalUUID#69798, systemId#69797]
(75) Project [codegen id : 9]
Output [1]: [internalUUID#69798]
Input [2]: [internalUUID#69798, systemId#69797]
(76) Exchange
Input [1]: [internalUUID#69798]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=52033]
(77) ShuffleQueryStage
Output [1]: [internalUUID#69798]
Arguments: 9
(78) BroadcastExchange
Input [1]: [internalUUID#69798]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=85527]
(79) BroadcastQueryStage
Output [1]: [internalUUID#69798]
Arguments: 143
(80) BroadcastHashJoin [codegen id : 73]
Left keys [2]: [coalesce(plant#69772, ), isnull(plant#69772)]
Right keys [2]: [coalesce(internalUUID#69798, ), isnull(internalUUID#69798)]
Join type: LeftOuter
Join condition: None
(81) Project [codegen id : 73]
Output [4]: [product#69771, plant#69772, _extract_code#71369, _extract_internalRefUUID#71370]
Input [5]: [product#69771, plant#69772, _extract_code#71369, _extract_internalRefUUID#71370, internalUUID#69798]
(82) ReusedExchange [Reuses operator id: 35]
Output [3]: [internalUUID#69826, baseUnitOfMeasure#69830, unitOfMeasureConversions#69832]
(83) BroadcastQueryStage
Output [3]: [internalUUID#69826, baseUnitOfMeasure#69830, unitOfMeasureConversions#69832]
Arguments: 108
(84) BroadcastHashJoin [codegen id : 73]
Left keys [1]: [product#69771]
Right keys [1]: [internalUUID#69826]
Join type: LeftOuter
Join condition: None
(85) Project
Output [2]: [plant#69772, filter(unitOfMeasureConversions#69832, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71369) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71370)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#69830.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69830.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#69849]
Input [7]: [product#69771, plant#69772, _extract_code#71369, _extract_internalRefUUID#71370, internalUUID#69826, baseUnitOfMeasure#69830, unitOfMeasureConversions#69832]
(86) Generate [codegen id : 74]
Input [2]: [plant#69772, filteredUnitOfMeasureConversions#69849]
Arguments: explode(filteredUnitOfMeasureConversions#69849.quantityNumerator), [plant#69772, filteredUnitOfMeasureConversions#69849], true, [quantityNumerator#69851]
(87) Project [codegen id : 74]
Output [2]: [plant#69772, filteredUnitOfMeasureConversions#69849.quantityDenominator AS _extract_quantityDenominator#70980]
Input [3]: [plant#69772, filteredUnitOfMeasureConversions#69849, quantityNumerator#69851]
(88) Generate [codegen id : 74]
Input [2]: [plant#69772, _extract_quantityDenominator#70980]
Arguments: explode(_extract_quantityDenominator#70980), [plant#69772], true, [quantityDenominator#69854]
(89) Project [codegen id : 74]
Output [1]: [plant#69772]
Input [2]: [plant#69772, quantityDenominator#69854]
(90) HashAggregate [codegen id : 74]
Input [1]: [plant#69772]
Keys [1]: [plant#69772]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69772]
(91) Exchange
Input [1]: [plant#69772]
Arguments: hashpartitioning(plant#69772, 25), ENSURE_REQUIREMENTS, [plan_id=110323]
(92) ShuffleQueryStage
Output [1]: [plant#69772]
Arguments: 152
(93) AQEShuffleRead
Input [1]: [plant#69772]
Arguments: coalesced
(94) HashAggregate
Input [1]: [plant#69772]
Keys [1]: [plant#69772]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69772]
(95) BroadcastHashJoin
Left keys [1]: [plantUuid#69186]
Right keys [1]: [plant#69772]
Join type: Inner
Join condition: None
(96) Project
Output [3]: [plantUuid#69186, storageLocId#69199, storageLocUuid#69200]
Input [4]: [plantUuid#69186, storageLocId#69199, storageLocUuid#69200, plant#69772]
(97) BroadcastHashJoin [codegen id : 83]
Left keys [2]: [coalesce(plant#69260.internalRefUUID, ), isnull(plant#69260.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#69186, ), isnull(plantUuid#69186)]
Join type: Inner
Join condition: None
(98) Generate
Input [9]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262, plantUuid#69186, storageLocId#69199, storageLocUuid#69200]
Arguments: explode(filter(stockQuantities#69262, lambdafunction(((lambda x#69269.storageLocation.internalRefUuid = storageLocUuid#69200) AND (lambda x#69269.storageLocation.Id = storageLocId#69199)), lambda x#69269, false))), [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, plantUuid#69186], false, [stockQuantity#69272]
(99) ObjectHashAggregate
Input [7]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, plantUuid#69186, stockQuantity#69272]
Keys [5]: [plantUuid#69186, internalUUID#69258, plant#69260, product#69259, systemId#69257]
Functions [2]: [partial_first(calculationDateTime#69261, false), partial_collect_list(stockQuantity#69272, 0, 0)]
Aggregate Attributes [3]: [first#73924, valueSet#73925, buf#73926]
Results [8]: [plantUuid#69186, internalUUID#69258, plant#69260, product#69259, systemId#69257, first#73927, valueSet#73928, buf#73929]
(100) ObjectHashAggregate
Input [8]: [plantUuid#69186, internalUUID#69258, plant#69260, product#69259, systemId#69257, first#73927, valueSet#73928, buf#73929]
Keys [5]: [plantUuid#69186, internalUUID#69258, plant#69260, product#69259, systemId#69257]
Functions [2]: [first(calculationDateTime#69261, false), collect_list(stockQuantity#69272, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#69261)()#69286, collect_list(stockQuantity#69272, 0, 0)#69287]
Results [4]: [product#69259, plant#69260, first(calculationDateTime#69261)()#69286 AS calculationDateTime#69273, collect_list(stockQuantity#69272, 0, 0)#69287 AS stockQuantities#69274]
(101) Filter [codegen id : 84]
Input [4]: [product#69259, plant#69260, calculationDateTime#69273, stockQuantities#69274]
Condition : ((size(stockQuantities#69274, true) > 0) AND isnotnull(calculationDateTime#69273))
(102) Generate [codegen id : 84]
Input [4]: [product#69259, plant#69260, calculationDateTime#69273, stockQuantities#69274]
Arguments: explode(stockQuantities#69274), [product#69259, plant#69260, calculationDateTime#69273], false, [stockQuantity#69289]
(103) Filter [codegen id : 84]
Input [4]: [product#69259, plant#69260, calculationDateTime#69273, stockQuantity#69289]
Condition : (((isnotnull(stockQuantity#69289.specialStockIndicator.code) AND isnotnull(stockQuantity#69289.stockType.code)) AND (stockQuantity#69289.specialStockIndicator.code = )) AND (stockQuantity#69289.stockType.code = 01))
(104) Project [codegen id : 84]
Output [4]: [plant#69260.internalRefUUID AS plantUuid#69291, product#69259.internalRefUUID AS productUuid#69293, calculationDateTime#69273, stockQuantity#69289.storagelocation.id AS storageLocId#69294]
Input [4]: [product#69259, plant#69260, calculationDateTime#69273, stockQuantity#69289]
(105) Exchange
Input [4]: [plantUuid#69291, productUuid#69293, calculationDateTime#69273, storageLocId#69294]
Arguments: hashpartitioning(productUuid#69293, plantUuid#69291, 25), ENSURE_REQUIREMENTS, [plan_id=115571]
(106) ShuffleQueryStage
Output [4]: [plantUuid#69291, productUuid#69293, calculationDateTime#69273, storageLocId#69294]
Arguments: 161
(107) AQEShuffleRead
Input [4]: [plantUuid#69291, productUuid#69293, calculationDateTime#69273, storageLocId#69294]
Arguments: local
(108) BroadcastExchange
Input [4]: [plantUuid#69291, productUuid#69293, calculationDateTime#69273, storageLocId#69294]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true], input[1, string, true]),false), [plan_id=117880]
(109) BroadcastQueryStage
Output [4]: [plantUuid#69291, productUuid#69293, calculationDateTime#69273, storageLocId#69294]
Arguments: 167
(110) Scan parquet
Output [3]: [product#69323, plant#69324, orderDateTime#69325]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(111) ColumnarToRow [codegen id : 11]
Input [3]: [product#69323, plant#69324, orderDateTime#69325]
(112) Filter [codegen id : 11]
Input [3]: [product#69323, plant#69324, orderDateTime#69325]
Condition : ((isnotnull(orderDateTime#69325) AND isnotnull(plant#69324)) AND isnotnull(product#69323))
(113) Exchange
Input [3]: [product#69323, plant#69324, orderDateTime#69325]
Arguments: hashpartitioning(product#69323, plant#69324, 25), REPARTITION_BY_NUM, [plan_id=52094]
(114) ShuffleQueryStage
Output [3]: [product#69323, plant#69324, orderDateTime#69325]
Arguments: 12
(115) ReusedExchange [Reuses operator id: 16]
Output [4]: [_extract_code#71375, _extract_internalRefUUID#71376, _extract_internalRefUUID#71378, _extract_internalRefUUID#71377]
(116) BroadcastQueryStage
Output [4]: [_extract_code#71375, _extract_internalRefUUID#71376, _extract_internalRefUUID#71378, _extract_internalRefUUID#71377]
Arguments: 126
(117) BroadcastHashJoin [codegen id : 91]
Left keys [4]: [coalesce(product#69323, ), isnull(product#69323), coalesce(plant#69324, ), isnull(plant#69324)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71378, ), isnull(_extract_internalRefUUID#71378), coalesce(_extract_internalRefUUID#71377, ), isnull(_extract_internalRefUUID#71377)]
Join type: LeftOuter
Join condition: None
(118) Project [codegen id : 91]
Output [5]: [product#69323, plant#69324, orderDateTime#69325, _extract_code#71375, _extract_internalRefUUID#71376]
Input [7]: [product#69323, plant#69324, orderDateTime#69325, _extract_code#71375, _extract_internalRefUUID#71376, _extract_internalRefUUID#71378, _extract_internalRefUUID#71377]
(119) ReusedExchange [Reuses operator id: 78]
Output [1]: [internalUUID#69338]
(120) BroadcastQueryStage
Output [1]: [internalUUID#69338]
Arguments: 144
(121) BroadcastHashJoin [codegen id : 91]
Left keys [2]: [coalesce(plant#69324, ), isnull(plant#69324)]
Right keys [2]: [coalesce(internalUUID#69338, ), isnull(internalUUID#69338)]
Join type: LeftOuter
Join condition: None
(122) Project [codegen id : 91]
Output [5]: [product#69323, plant#69324, orderDateTime#69325, _extract_code#71375, _extract_internalRefUUID#71376]
Input [6]: [product#69323, plant#69324, orderDateTime#69325, _extract_code#71375, _extract_internalRefUUID#71376, internalUUID#69338]
(123) ReusedExchange [Reuses operator id: 35]
Output [3]: [internalUUID#69352, baseUnitOfMeasure#69356, unitOfMeasureConversions#69358]
(124) BroadcastQueryStage
Output [3]: [internalUUID#69352, baseUnitOfMeasure#69356, unitOfMeasureConversions#69358]
Arguments: 109
(125) BroadcastHashJoin [codegen id : 91]
Left keys [1]: [product#69323]
Right keys [1]: [internalUUID#69352]
Join type: LeftOuter
Join condition: None
(126) Project
Output [4]: [product#69323, plant#69324, orderDateTime#69325, filter(unitOfMeasureConversions#69358, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71375) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71376)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#69356.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69356.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#69363]
Input [8]: [product#69323, plant#69324, orderDateTime#69325, _extract_code#71375, _extract_internalRefUUID#71376, internalUUID#69352, baseUnitOfMeasure#69356, unitOfMeasureConversions#69358]
(127) Generate
Input [4]: [product#69323, plant#69324, orderDateTime#69325, filteredUnitOfMeasureConversions#69363]
Arguments: explode(filteredUnitOfMeasureConversions#69363.quantityNumerator), [product#69323, plant#69324, orderDateTime#69325, filteredUnitOfMeasureConversions#69363], true, [quantityNumerator#69364]
(128) Project
Output [4]: [product#69323, plant#69324, orderDateTime#69325, filteredUnitOfMeasureConversions#69363.quantityDenominator AS _extract_quantityDenominator#70990]
Input [5]: [product#69323, plant#69324, orderDateTime#69325, filteredUnitOfMeasureConversions#69363, quantityNumerator#69364]
(129) Generate
Input [4]: [product#69323, plant#69324, orderDateTime#69325, _extract_quantityDenominator#70990]
Arguments: explode(_extract_quantityDenominator#70990), [product#69323, plant#69324, orderDateTime#69325], true, [quantityDenominator#69365]
(130) Project
Output [3]: [product#69323, plant#69324, orderDateTime#69325]
Input [4]: [product#69323, plant#69324, orderDateTime#69325, quantityDenominator#69365]
(131) BroadcastHashJoin [codegen id : 92]
Left keys [2]: [plantUuid#69291, productUuid#69293]
Right keys [2]: [plant#69324, product#69323]
Join type: Inner
Join condition: (calculationDateTime#69273 <= orderDateTime#69325)
(132) Project [codegen id : 92]
Output [4]: [plantUuid#69291 AS plant#69435, productUuid#69293 AS product#69436, storageLocId#69294, calculationDateTime#69273]
Input [7]: [plantUuid#69291, productUuid#69293, calculationDateTime#69273, storageLocId#69294, product#69323, plant#69324, orderDateTime#69325]
(133) Exchange
Input [4]: [plant#69435, product#69436, storageLocId#69294, calculationDateTime#69273]
Arguments: hashpartitioning(coalesce(plant#69435, ), isnull(plant#69435), coalesce(product#69436, ), isnull(product#69436), coalesce(storageLocId#69294, ), isnull(storageLocId#69294), coalesce(calculationDateTime#69273, 1970-01-01 00:00:00), isnull(calculationDateTime#69273), 25), ENSURE_REQUIREMENTS, [plan_id=119285]
(134) ShuffleQueryStage
Output [4]: [plant#69435, product#69436, storageLocId#69294, calculationDateTime#69273]
Arguments: 173
(135) AQEShuffleRead
Input [4]: [plant#69435, product#69436, storageLocId#69294, calculationDateTime#69273]
Arguments: local
(136) BroadcastExchange
Input [4]: [plant#69435, product#69436, storageLocId#69294, calculationDateTime#69273]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true]), coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, timestamp, true], 1970-01-01 00:00:00), isnull(input[3, timestamp, true])),false), [plan_id=123597]
(137) BroadcastQueryStage
Output [4]: [plant#69435, product#69436, storageLocId#69294, calculationDateTime#69273]
Arguments: 179
(138) ReusedExchange [Reuses operator id: 108]
Output [4]: [plantUuid#69540, productUuid#69542, calculationDateTime#69536, storageLocId#69543]
(139) BroadcastQueryStage
Output [4]: [plantUuid#69540, productUuid#69542, calculationDateTime#69536, storageLocId#69543]
Arguments: 169
(140) ReusedExchange [Reuses operator id: 113]
Output [3]: [product#69548, plant#69549, orderDateTime#69550]
(141) ShuffleQueryStage
Output [3]: [product#69548, plant#69549, orderDateTime#69550]
Arguments: 32
(142) ReusedExchange [Reuses operator id: 16]
Output [4]: [_extract_code#71393, _extract_internalRefUUID#71394, _extract_internalRefUUID#71391, _extract_internalRefUUID#71392]
(143) BroadcastQueryStage
Output [4]: [_extract_code#71393, _extract_internalRefUUID#71394, _extract_internalRefUUID#71391, _extract_internalRefUUID#71392]
Arguments: 130
(144) BroadcastHashJoin [codegen id : 93]
Left keys [4]: [coalesce(product#69548, ), isnull(product#69548), coalesce(plant#69549, ), isnull(plant#69549)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71391, ), isnull(_extract_internalRefUUID#71391), coalesce(_extract_internalRefUUID#71392, ), isnull(_extract_internalRefUUID#71392)]
Join type: LeftOuter
Join condition: None
(145) Project [codegen id : 93]
Output [5]: [product#69548, plant#69549, orderDateTime#69550, _extract_code#71393, _extract_internalRefUUID#71394]
Input [7]: [product#69548, plant#69549, orderDateTime#69550, _extract_code#71393, _extract_internalRefUUID#71394, _extract_internalRefUUID#71391, _extract_internalRefUUID#71392]
(146) ReusedExchange [Reuses operator id: 78]
Output [1]: [internalUUID#69562]
(147) BroadcastQueryStage
Output [1]: [internalUUID#69562]
Arguments: 146
(148) BroadcastHashJoin [codegen id : 93]
Left keys [2]: [coalesce(plant#69549, ), isnull(plant#69549)]
Right keys [2]: [coalesce(internalUUID#69562, ), isnull(internalUUID#69562)]
Join type: LeftOuter
Join condition: None
(149) Project [codegen id : 93]
Output [5]: [product#69548, plant#69549, orderDateTime#69550, _extract_code#71393, _extract_internalRefUUID#71394]
Input [6]: [product#69548, plant#69549, orderDateTime#69550, _extract_code#71393, _extract_internalRefUUID#71394, internalUUID#69562]
(150) ReusedExchange [Reuses operator id: 35]
Output [3]: [internalUUID#69576, baseUnitOfMeasure#69580, unitOfMeasureConversions#69582]
(151) BroadcastQueryStage
Output [3]: [internalUUID#69576, baseUnitOfMeasure#69580, unitOfMeasureConversions#69582]
Arguments: 111
(152) BroadcastHashJoin [codegen id : 93]
Left keys [1]: [product#69548]
Right keys [1]: [internalUUID#69576]
Join type: LeftOuter
Join condition: None
(153) Project
Output [4]: [product#69548, plant#69549, orderDateTime#69550, filter(unitOfMeasureConversions#69582, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71393) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71394)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#69580.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69580.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#69587]
Input [8]: [product#69548, plant#69549, orderDateTime#69550, _extract_code#71393, _extract_internalRefUUID#71394, internalUUID#69576, baseUnitOfMeasure#69580, unitOfMeasureConversions#69582]
(154) Generate
Input [4]: [product#69548, plant#69549, orderDateTime#69550, filteredUnitOfMeasureConversions#69587]
Arguments: explode(filteredUnitOfMeasureConversions#69587.quantityNumerator), [product#69548, plant#69549, orderDateTime#69550, filteredUnitOfMeasureConversions#69587], true, [quantityNumerator#69588]
(155) Project
Output [4]: [product#69548, plant#69549, orderDateTime#69550, filteredUnitOfMeasureConversions#69587.quantityDenominator AS _extract_quantityDenominator#71011]
Input [5]: [product#69548, plant#69549, orderDateTime#69550, filteredUnitOfMeasureConversions#69587, quantityNumerator#69588]
(156) Generate
Input [4]: [product#69548, plant#69549, orderDateTime#69550, _extract_quantityDenominator#71011]
Arguments: explode(_extract_quantityDenominator#71011), [product#69548, plant#69549, orderDateTime#69550], true, [quantityDenominator#69589]
(157) Project
Output [3]: [product#69548, plant#69549, orderDateTime#69550]
Input [4]: [product#69548, plant#69549, orderDateTime#69550, quantityDenominator#69589]
(158) BroadcastHashJoin [codegen id : 94]
Left keys [2]: [plantUuid#69540, productUuid#69542]
Right keys [2]: [plant#69549, product#69548]
Join type: Inner
Join condition: (calculationDateTime#69536 <= orderDateTime#69550)
(159) Project [codegen id : 94]
Output [4]: [plantUuid#69540 AS plant#69620, productUuid#69542 AS product#69621, storageLocId#69543, calculationDateTime#69536]
Input [7]: [plantUuid#69540, productUuid#69542, calculationDateTime#69536, storageLocId#69543, product#69548, plant#69549, orderDateTime#69550]
(160) HashAggregate [codegen id : 94]
Input [4]: [plant#69620, product#69621, storageLocId#69543, calculationDateTime#69536]
Keys [3]: [plant#69620, product#69621, storageLocId#69543]
Functions [1]: [partial_max(calculationDateTime#69536)]
Aggregate Attributes [1]: [max#73930]
Results [4]: [plant#69620, product#69621, storageLocId#69543, max#73931]
(161) HashAggregate [codegen id : 94]
Input [4]: [plant#69620, product#69621, storageLocId#69543, max#73931]
Keys [3]: [plant#69620, product#69621, storageLocId#69543]
Functions [1]: [max(calculationDateTime#69536)]
Aggregate Attributes [1]: [max(calculationDateTime#69536)#69447]
Results [4]: [plant#69620, product#69621, storageLocId#69543, max(calculationDateTime#69536)#69447 AS max_calc_datetime#69441]
(162) Exchange
Input [4]: [plant#69620, product#69621, storageLocId#69543, max_calc_datetime#69441]
Arguments: hashpartitioning(coalesce(plant#69620, ), isnull(plant#69620), coalesce(product#69621, ), isnull(product#69621), coalesce(storageLocId#69543, ), isnull(storageLocId#69543), coalesce(max_calc_datetime#69441, 1970-01-01 00:00:00), isnull(max_calc_datetime#69441), 25), ENSURE_REQUIREMENTS, [plan_id=119344]
(163) ShuffleQueryStage
Output [4]: [plant#69620, product#69621, storageLocId#69543, max_calc_datetime#69441]
Arguments: 174
(164) AQEShuffleRead
Input [4]: [plant#69620, product#69621, storageLocId#69543, max_calc_datetime#69441]
Arguments: local
(165) BroadcastHashJoin [codegen id : 99]
Left keys [8]: [coalesce(plant#69435, ), isnull(plant#69435), coalesce(product#69436, ), isnull(product#69436), coalesce(storageLocId#69294, ), isnull(storageLocId#69294), coalesce(calculationDateTime#69273, 1970-01-01 00:00:00), isnull(calculationDateTime#69273)]
Right keys [8]: [coalesce(plant#69620, ), isnull(plant#69620), coalesce(product#69621, ), isnull(product#69621), coalesce(storageLocId#69543, ), isnull(storageLocId#69543), coalesce(max_calc_datetime#69441, 1970-01-01 00:00:00), isnull(max_calc_datetime#69441)]
Join type: Inner
Join condition: None
(166) Project [codegen id : 99]
Output [3]: [plant#69435, product#69436, calculationDateTime#69273]
Input [8]: [plant#69435, product#69436, storageLocId#69294, calculationDateTime#69273, plant#69620, product#69621, storageLocId#69543, max_calc_datetime#69441]
(167) HashAggregate [codegen id : 99]
Input [3]: [plant#69435, product#69436, calculationDateTime#69273]
Keys [2]: [product#69436, plant#69435]
Functions [1]: [partial_min(calculationDateTime#69273)]
Aggregate Attributes [1]: [min#73914]
Results [3]: [product#69436, plant#69435, min#73915]
(168) Exchange
Input [3]: [product#69436, plant#69435, min#73915]
Arguments: hashpartitioning(product#69436, plant#69435, 25), ENSURE_REQUIREMENTS, [plan_id=124704]
(169) ShuffleQueryStage
Output [3]: [product#69436, plant#69435, min#73915]
Arguments: 180
(170) AQEShuffleRead
Input [3]: [product#69436, plant#69435, min#73915]
Arguments: coalesced
(171) HashAggregate [codegen id : 101]
Input [3]: [product#69436, plant#69435, min#73915]
Keys [2]: [product#69436, plant#69435]
Functions [1]: [min(calculationDateTime#69273)]
Aggregate Attributes [1]: [min(calculationDateTime#69273)#69766]
Results [3]: [product#69436, plant#69435, min(calculationDateTime#69273)#69766 AS minCalculationDateTime#69752]
(172) BroadcastExchange
Input [3]: [product#69436, plant#69435, minCalculationDateTime#69752]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=126902]
(173) BroadcastQueryStage
Output [3]: [product#69436, plant#69435, minCalculationDateTime#69752]
Arguments: 182
(174) BroadcastHashJoin [codegen id : 104]
Left keys [2]: [plant#69020, product#69019]
Right keys [2]: [plant#69435, product#69436]
Join type: LeftOuter
Join condition: None
(175) Project [codegen id : 104]
Output [4]: [product#69019, plant#69020, timezonecode#69079, CASE WHEN isnotnull(minCalculationDateTime#69752) THEN minCalculationDateTime#69752 ELSE orderDateTime#69021 END AS stockDatetime#69868]
Input [7]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, product#69436, plant#69435, minCalculationDateTime#69752]
(176) Exchange
Input [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: hashpartitioning(coalesce(plant#69020, ), isnull(plant#69020), coalesce(product#69019, ), isnull(product#69019), 25), ENSURE_REQUIREMENTS, [plan_id=128012]
(177) ShuffleQueryStage
Output [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: 184
(178) AQEShuffleRead
Input [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: local
(179) Scan parquet
Output [3]: [product#69978, plant#69979, orderDateTime#69980]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(180) ColumnarToRow [codegen id : 25]
Input [3]: [product#69978, plant#69979, orderDateTime#69980]
(181) Exchange
Input [3]: [product#69978, plant#69979, orderDateTime#69980]
Arguments: hashpartitioning(product#69978, plant#69979, 25), REPARTITION_BY_NUM, [plan_id=52437]
(182) ShuffleQueryStage
Output [3]: [product#69978, plant#69979, orderDateTime#69980]
Arguments: 39
(183) Scan parquet
Output [8]: [internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989, systemId#69982]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>
(184) ColumnarToRow [codegen id : 26]
Input [8]: [internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989, systemId#69982]
(185) Project [codegen id : 26]
Output [8]: [systemId#69982, internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989]
Input [8]: [internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989, systemId#69982]
(186) Exchange
Input [8]: [systemId#69982, internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=52451]
(187) ShuffleQueryStage
Output [8]: [systemId#69982, internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989]
Arguments: 40
(188) Project [codegen id : 71]
Output [1]: [struct(systemId, systemId#69982, internalUUID, internalUUID#69983, plant, plant#69984, product, product#69985, blockedForReplenishmentStartingFrom, blockedForReplenishmentStartingFrom#69986, productionAspect, productionAspect#69987, salesPlant, salesPlant#69988, listing, listing#69989) AS productPlant#69990]
Input [8]: [systemId#69982, internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989]
(189) BroadcastExchange
Input [1]: [productPlant#69990]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID), coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID)),false), [plan_id=78088]
(190) BroadcastQueryStage
Output [1]: [productPlant#69990]
Arguments: 141
(191) BroadcastHashJoin [codegen id : 105]
Left keys [4]: [coalesce(product#69978, ), isnull(product#69978), coalesce(plant#69979, ), isnull(plant#69979)]
Right keys [4]: [coalesce(productPlant#69990.product.internalRefUUID, ), isnull(productPlant#69990.product.internalRefUUID), coalesce(productPlant#69990.plant.internalRefUUID, ), isnull(productPlant#69990.plant.internalRefUUID)]
Join type: LeftOuter
Join condition: None
(192) ReusedExchange [Reuses operator id: 25]
Output [2]: [internalUUID#69992, timezonecode#70003]
(193) BroadcastQueryStage
Output [2]: [internalUUID#69992, timezonecode#70003]
Arguments: 106
(194) BroadcastHashJoin [codegen id : 105]
Left keys [2]: [coalesce(plant#69979, ), isnull(plant#69979)]
Right keys [2]: [coalesce(internalUUID#69992, ), isnull(internalUUID#69992)]
Join type: LeftOuter
Join condition: None
(195) Project [codegen id : 105]
Output [5]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, CASE WHEN isnotnull(timezonecode#70003) THEN timezonecode#70003 ELSE UTC END AS timezonecode#70005]
Input [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, internalUUID#69992, timezonecode#70003]
(196) ReusedExchange [Reuses operator id: 35]
Output [3]: [internalUUID#70007, baseUnitOfMeasure#70011, unitOfMeasureConversions#70013]
(197) BroadcastQueryStage
Output [3]: [internalUUID#70007, baseUnitOfMeasure#70011, unitOfMeasureConversions#70013]
Arguments: 112
(198) BroadcastHashJoin [codegen id : 105]
Left keys [1]: [product#69978]
Right keys [1]: [internalUUID#70007]
Join type: LeftOuter
Join condition: None
(199) Project
Output [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, filter(unitOfMeasureConversions#70013, lambdafunction((((lambda u#69109.measurementUnit1.code = productPlant#69990.productionAspect.productMovementPlants.unitOfIssue.code) AND (lambda u#69109.measurementUnit1.internalRefUUID = productPlant#69990.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#70011.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70011.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#70018]
Input [8]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, internalUUID#70007, baseUnitOfMeasure#70011, unitOfMeasureConversions#70013]
(200) Project [codegen id : 106]
Output [7]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, filteredUnitOfMeasureConversions#70018.quantityDenominator AS _extract_quantityDenominator#71133, filteredUnitOfMeasureConversions#70018.quantityNumerator AS _extract_quantityNumerator#71134]
Input [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, filteredUnitOfMeasureConversions#70018]
(201) Generate [codegen id : 106]
Input [7]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, _extract_quantityDenominator#71133, _extract_quantityNumerator#71134]
Arguments: explode(_extract_quantityNumerator#71134), [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, _extract_quantityDenominator#71133], true, [quantityNumerator#70019]
(202) Generate [codegen id : 106]
Input [7]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, _extract_quantityDenominator#71133, quantityNumerator#70019]
Arguments: explode(_extract_quantityDenominator#71133), [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, quantityNumerator#70019], true, [quantityDenominator#70020]
(203) Project [codegen id : 106]
Output [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, (cast(quantityNumerator#70019 as double) / cast(quantityDenominator#70020 as double)) AS outboundUnit#70021]
Input [7]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, quantityNumerator#70019, quantityDenominator#70020]
(204) Project [codegen id : 106]
Output [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, CASE WHEN (isnull(outboundUnit#70021) OR (outboundUnit#70021 <= 0.0)) THEN 1.0 ELSE outboundUnit#70021 END AS outboundUnit#70022]
Input [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70021]
(205) ReusedExchange [Reuses operator id: 49]
Output [6]: [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, stockQuantities#70028]
(206) BroadcastQueryStage
Output [6]: [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, stockQuantities#70028]
Arguments: 120
(207) ReusedExchange [Reuses operator id: 62]
Output [3]: [plantUuid#70071, storageLocId#70072, storageLocUuid#70073]
(208) BroadcastQueryStage
Output [3]: [plantUuid#70071, storageLocId#70072, storageLocUuid#70073]
Arguments: 101
(209) ReusedExchange [Reuses operator id: 91]
Output [1]: [plant#70075]
(210) ShuffleQueryStage
Output [1]: [plant#70075]
Arguments: 156
(211) AQEShuffleRead
Input [1]: [plant#70075]
Arguments: coalesced
(212) HashAggregate
Input [1]: [plant#70075]
Keys [1]: [plant#70075]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70075]
(213) BroadcastHashJoin
Left keys [1]: [plantUuid#70071]
Right keys [1]: [plant#70075]
Join type: Inner
Join condition: None
(214) Project
Output [3]: [plantUuid#70071, storageLocId#70072, storageLocUuid#70073]
Input [4]: [plantUuid#70071, storageLocId#70072, storageLocUuid#70073, plant#70075]
(215) BroadcastHashJoin [codegen id : 87]
Left keys [2]: [coalesce(plant#70026.internalRefUUID, ), isnull(plant#70026.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#70071, ), isnull(plantUuid#70071)]
Join type: Inner
Join condition: None
(216) Generate
Input [9]: [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, stockQuantities#70028, plantUuid#70071, storageLocId#70072, storageLocUuid#70073]
Arguments: explode(filter(stockQuantities#70028, lambdafunction(((lambda x#69269.storageLocation.internalRefUuid = storageLocUuid#70073) AND (lambda x#69269.storageLocation.Id = storageLocId#70072)), lambda x#69269, false))), [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, plantUuid#70071], false, [stockQuantity#70118]
(217) ObjectHashAggregate
Input [7]: [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, plantUuid#70071, stockQuantity#70118]
Keys [5]: [plantUuid#70071, internalUUID#70024, plant#70026, product#70025, systemId#70023]
Functions [2]: [partial_first(calculationDateTime#70027, false), partial_collect_list(stockQuantity#70118, 0, 0)]
Aggregate Attributes [3]: [first#73962, valueSet#73963, buf#73964]
Results [8]: [plantUuid#70071, internalUUID#70024, plant#70026, product#70025, systemId#70023, first#73965, valueSet#73966, buf#73967]
(218) ObjectHashAggregate
Input [8]: [plantUuid#70071, internalUUID#70024, plant#70026, product#70025, systemId#70023, first#73965, valueSet#73966, buf#73967]
Keys [5]: [plantUuid#70071, internalUUID#70024, plant#70026, product#70025, systemId#70023]
Functions [2]: [first(calculationDateTime#70027, false), collect_list(stockQuantity#70118, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#70027)()#69286, collect_list(stockQuantity#70118, 0, 0)#69287]
Results [4]: [product#70025, plant#70026, first(calculationDateTime#70027)()#69286 AS calculationDateTime#70119, collect_list(stockQuantity#70118, 0, 0)#69287 AS stockQuantities#70120]
(219) Filter [codegen id : 88]
Input [4]: [product#70025, plant#70026, calculationDateTime#70119, stockQuantities#70120]
Condition : ((size(stockQuantities#70120, true) > 0) AND isnotnull(calculationDateTime#70119))
(220) Generate [codegen id : 88]
Input [4]: [product#70025, plant#70026, calculationDateTime#70119, stockQuantities#70120]
Arguments: explode(stockQuantities#70120), [product#70025, plant#70026, calculationDateTime#70119], false, [stockQuantity#70121]
(221) Filter [codegen id : 88]
Input [4]: [product#70025, plant#70026, calculationDateTime#70119, stockQuantity#70121]
Condition : (((isnotnull(stockQuantity#70121.specialStockIndicator.code) AND isnotnull(stockQuantity#70121.stockType.code)) AND (stockQuantity#70121.specialStockIndicator.code = )) AND (stockQuantity#70121.stockType.code = 01))
(222) Project [codegen id : 88]
Output [5]: [plant#70026.internalRefUUID AS plantUuid#70123, product#70025.internalRefUUID AS productUuid#70125, calculationDateTime#70119, stockQuantity#70121.storagelocation.id AS storageLocId#70126, stockQuantity#70121.quantity.measure AS quantity#70128]
Input [4]: [product#70025, plant#70026, calculationDateTime#70119, stockQuantity#70121]
(223) Exchange
Input [5]: [plantUuid#70123, productUuid#70125, calculationDateTime#70119, storageLocId#70126, quantity#70128]
Arguments: hashpartitioning(productUuid#70125, plantUuid#70123, 25), ENSURE_REQUIREMENTS, [plan_id=115721]
(224) ShuffleQueryStage
Output [5]: [plantUuid#70123, productUuid#70125, calculationDateTime#70119, storageLocId#70126, quantity#70128]
Arguments: 164
(225) AQEShuffleRead
Input [5]: [plantUuid#70123, productUuid#70125, calculationDateTime#70119, storageLocId#70126, quantity#70128]
Arguments: local
(226) BroadcastExchange
Input [5]: [plantUuid#70123, productUuid#70125, calculationDateTime#70119, storageLocId#70126, quantity#70128]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true], input[1, string, true]),false), [plan_id=118595]
(227) BroadcastQueryStage
Output [5]: [plantUuid#70123, productUuid#70125, calculationDateTime#70119, storageLocId#70126, quantity#70128]
Arguments: 172
(228) ReusedExchange [Reuses operator id: 113]
Output [3]: [product#70131, plant#70132, orderDateTime#70133]
(229) ShuffleQueryStage
Output [3]: [product#70131, plant#70132, orderDateTime#70133]
Arguments: 58
(230) ReusedExchange [Reuses operator id: 16]
Output [4]: [_extract_code#71408, _extract_internalRefUUID#71409, _extract_internalRefUUID#71410, _extract_internalRefUUID#71407]
(231) BroadcastQueryStage
Output [4]: [_extract_code#71408, _extract_internalRefUUID#71409, _extract_internalRefUUID#71410, _extract_internalRefUUID#71407]
Arguments: 134
(232) BroadcastHashJoin [codegen id : 97]
Left keys [4]: [coalesce(product#70131, ), isnull(product#70131), coalesce(plant#70132, ), isnull(plant#70132)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71410, ), isnull(_extract_internalRefUUID#71410), coalesce(_extract_internalRefUUID#71407, ), isnull(_extract_internalRefUUID#71407)]
Join type: LeftOuter
Join condition: None
(233) Project [codegen id : 97]
Output [5]: [product#70131, plant#70132, orderDateTime#70133, _extract_code#71408, _extract_internalRefUUID#71409]
Input [7]: [product#70131, plant#70132, orderDateTime#70133, _extract_code#71408, _extract_internalRefUUID#71409, _extract_internalRefUUID#71410, _extract_internalRefUUID#71407]
(234) ReusedExchange [Reuses operator id: 78]
Output [1]: [internalUUID#70145]
(235) BroadcastQueryStage
Output [1]: [internalUUID#70145]
Arguments: 148
(236) BroadcastHashJoin [codegen id : 97]
Left keys [2]: [coalesce(plant#70132, ), isnull(plant#70132)]
Right keys [2]: [coalesce(internalUUID#70145, ), isnull(internalUUID#70145)]
Join type: LeftOuter
Join condition: None
(237) Project [codegen id : 97]
Output [5]: [product#70131, plant#70132, orderDateTime#70133, _extract_code#71408, _extract_internalRefUUID#71409]
Input [6]: [product#70131, plant#70132, orderDateTime#70133, _extract_code#71408, _extract_internalRefUUID#71409, internalUUID#70145]
(238) ReusedExchange [Reuses operator id: 35]
Output [3]: [internalUUID#70159, baseUnitOfMeasure#70163, unitOfMeasureConversions#70165]
(239) BroadcastQueryStage
Output [3]: [internalUUID#70159, baseUnitOfMeasure#70163, unitOfMeasureConversions#70165]
Arguments: 114
(240) BroadcastHashJoin [codegen id : 97]
Left keys [1]: [product#70131]
Right keys [1]: [internalUUID#70159]
Join type: LeftOuter
Join condition: None
(241) Project
Output [4]: [product#70131, plant#70132, orderDateTime#70133, filter(unitOfMeasureConversions#70165, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71408) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71409)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#70163.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70163.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#70171]
Input [8]: [product#70131, plant#70132, orderDateTime#70133, _extract_code#71408, _extract_internalRefUUID#71409, internalUUID#70159, baseUnitOfMeasure#70163, unitOfMeasureConversions#70165]
(242) Generate
Input [4]: [product#70131, plant#70132, orderDateTime#70133, filteredUnitOfMeasureConversions#70171]
Arguments: explode(filteredUnitOfMeasureConversions#70171.quantityNumerator), [product#70131, plant#70132, orderDateTime#70133, filteredUnitOfMeasureConversions#70171], true, [quantityNumerator#70172]
(243) Project
Output [4]: [product#70131, plant#70132, orderDateTime#70133, filteredUnitOfMeasureConversions#70171.quantityDenominator AS _extract_quantityDenominator#71031]
Input [5]: [product#70131, plant#70132, orderDateTime#70133, filteredUnitOfMeasureConversions#70171, quantityNumerator#70172]
(244) Generate
Input [4]: [product#70131, plant#70132, orderDateTime#70133, _extract_quantityDenominator#71031]
Arguments: explode(_extract_quantityDenominator#71031), [product#70131, plant#70132, orderDateTime#70133], true, [quantityDenominator#70173]
(245) Project
Output [3]: [product#70131, plant#70132, orderDateTime#70133]
Input [4]: [product#70131, plant#70132, orderDateTime#70133, quantityDenominator#70173]
(246) BroadcastHashJoin [codegen id : 98]
Left keys [2]: [plantUuid#70123, productUuid#70125]
Right keys [2]: [plant#70132, product#70131]
Join type: Inner
Join condition: (calculationDateTime#70119 <= orderDateTime#70133)
(247) Project [codegen id : 98]
Output [5]: [plantUuid#70123 AS plant#70176, productUuid#70125 AS product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128]
Input [8]: [plantUuid#70123, productUuid#70125, calculationDateTime#70119, storageLocId#70126, quantity#70128, product#70131, plant#70132, orderDateTime#70133]
(248) Exchange
Input [5]: [plant#70176, product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128]
Arguments: hashpartitioning(coalesce(plant#70176, ), isnull(plant#70176), coalesce(product#70177, ), isnull(product#70177), coalesce(storageLocId#70126, ), isnull(storageLocId#70126), coalesce(calculationDateTime#70119, 1970-01-01 00:00:00), isnull(calculationDateTime#70119), 25), ENSURE_REQUIREMENTS, [plan_id=119911]
(249) ShuffleQueryStage
Output [5]: [plant#70176, product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128]
Arguments: 177
(250) AQEShuffleRead
Input [5]: [plant#70176, product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128]
Arguments: local
(251) BroadcastExchange
Input [5]: [plant#70176, product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true]), coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, timestamp, true], 1970-01-01 00:00:00), isnull(input[3, timestamp, true])),false), [plan_id=123247]
(252) BroadcastQueryStage
Output [5]: [plant#70176, product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128]
Arguments: 178
(253) ReusedExchange [Reuses operator id: 162]
Output [4]: [plant#70323, product#70324, storageLocId#70264, max_calc_datetime#70325]
(254) ShuffleQueryStage
Output [4]: [plant#70323, product#70324, storageLocId#70264, max_calc_datetime#70325]
Arguments: 176
(255) AQEShuffleRead
Input [4]: [plant#70323, product#70324, storageLocId#70264, max_calc_datetime#70325]
Arguments: local
(256) BroadcastHashJoin [codegen id : 100]
Left keys [8]: [coalesce(plant#70176, ), isnull(plant#70176), coalesce(product#70177, ), isnull(product#70177), coalesce(storageLocId#70126, ), isnull(storageLocId#70126), coalesce(calculationDateTime#70119, 1970-01-01 00:00:00), isnull(calculationDateTime#70119)]
Right keys [8]: [coalesce(plant#70323, ), isnull(plant#70323), coalesce(product#70324, ), isnull(product#70324), coalesce(storageLocId#70264, ), isnull(storageLocId#70264), coalesce(max_calc_datetime#70325, 1970-01-01 00:00:00), isnull(max_calc_datetime#70325)]
Join type: Inner
Join condition: None
(257) Project [codegen id : 100]
Output [4]: [plant#70176, product#70177, calculationDateTime#70119, quantity#70128]
Input [9]: [plant#70176, product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128, plant#70323, product#70324, storageLocId#70264, max_calc_datetime#70325]
(258) HashAggregate [codegen id : 100]
Input [4]: [plant#70176, product#70177, calculationDateTime#70119, quantity#70128]
Keys [2]: [product#70177, plant#70176]
Functions [2]: [partial_min(calculationDateTime#70119), partial_sum(quantity#70128)]
Aggregate Attributes [2]: [min#73956, sum#73957]
Results [4]: [product#70177, plant#70176, min#73958, sum#73959]
(259) Exchange
Input [4]: [product#70177, plant#70176, min#73958, sum#73959]
Arguments: hashpartitioning(product#70177, plant#70176, 25), ENSURE_REQUIREMENTS, [plan_id=124782]
(260) ShuffleQueryStage
Output [4]: [product#70177, plant#70176, min#73958, sum#73959]
Arguments: 181
(261) AQEShuffleRead
Input [4]: [product#70177, plant#70176, min#73958, sum#73959]
Arguments: coalesced
(262) HashAggregate [codegen id : 102]
Input [4]: [product#70177, plant#70176, min#73958, sum#73959]
Keys [2]: [product#70177, plant#70176]
Functions [2]: [min(calculationDateTime#70119), sum(quantity#70128)]
Aggregate Attributes [2]: [min(calculationDateTime#70119)#69766, sum(quantity#70128)#69765]
Results [4]: [product#70177, plant#70176, min(calculationDateTime#70119)#69766 AS minCalculationDateTime#70327, round(sum(quantity#70128)#69765, 0) AS roundedQuantitySum#70328]
(263) BroadcastExchange
Input [4]: [product#70177, plant#70176, minCalculationDateTime#70327, roundedQuantitySum#70328]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true], input[0, string, true]),false), [plan_id=127175]
(264) BroadcastQueryStage
Output [4]: [product#70177, plant#70176, minCalculationDateTime#70327, roundedQuantitySum#70328]
Arguments: 183
(265) BroadcastHashJoin [codegen id : 106]
Left keys [2]: [plant#69979, product#69978]
Right keys [2]: [plant#70176, product#70177]
Join type: LeftOuter
Join condition: None
(266) Project [codegen id : 106]
Output [8]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, CASE WHEN isnotnull(minCalculationDateTime#70327) THEN minCalculationDateTime#70327 ELSE orderDateTime#69980 END AS stockDatetime#70329, CASE WHEN isnotnull(roundedQuantitySum#70328) THEN array(roundedQuantitySum#70328) ELSE [0.0] END AS stockBatchQuantities#70330]
Input [10]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, product#70177, plant#70176, minCalculationDateTime#70327, roundedQuantitySum#70328]
(267) Scan parquet
Output [9]: [product#69873, plant#69874, demandChannel#69875, demandStream#69876, considerVariance#69877, demandTimeBuckets#69878, demandPointInTimeStart#69879, demandPointInTimeEnd#69880, demandPointInTime#69881]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-demand-service/out/demand/10000002106/0_1_10000002106]
PushedFilters: [IsNotNull(product), IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string,demandChannel:string,demandStream:string,considerVariance:boolean,demandTimeBuckets:array<struct<demandStartDateTime:timestamp,demandQuantity:double,demandVariance:double>>,demandPointInTimeStart:timestamp,demandPointInTimeEnd:timestamp,demandPointInTime:array<struct<demandDateTime:timestamp,demandQuantity:double,demandVariance:double>>>
(268) ColumnarToRow [codegen id : 49]
Input [9]: [product#69873, plant#69874, demandChannel#69875, demandStream#69876, considerVariance#69877, demandTimeBuckets#69878, demandPointInTimeStart#69879, demandPointInTimeEnd#69880, demandPointInTime#69881]
(269) Filter [codegen id : 49]
Input [9]: [product#69873, plant#69874, demandChannel#69875, demandStream#69876, considerVariance#69877, demandTimeBuckets#69878, demandPointInTimeStart#69879, demandPointInTimeEnd#69880, demandPointInTime#69881]
Condition : (isnotnull(product#69873) AND isnotnull(plant#69874))
(270) BroadcastExchange
Input [9]: [product#69873, plant#69874, demandChannel#69875, demandStream#69876, considerVariance#69877, demandTimeBuckets#69878, demandPointInTimeStart#69879, demandPointInTimeEnd#69880, demandPointInTime#69881]
Arguments: HashedRelationBroadcastMode(List(input[0, string, false], input[1, string, false]),false), [plan_id=53015]
(271) BroadcastQueryStage
Output [9]: [product#69873, plant#69874, demandChannel#69875, demandStream#69876, considerVariance#69877, demandTimeBuckets#69878, demandPointInTimeStart#69879, demandPointInTimeEnd#69880, demandPointInTime#69881]
Arguments: 85
(272) BroadcastHashJoin [codegen id : 106]
Left keys [2]: [product#69978, plant#69979]
Right keys [2]: [product#69873, plant#69874]
Join type: LeftOuter
Join condition: None
(273) Project [codegen id : 106]
Output [9]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, CASE WHEN isnotnull(product#69873) THEN struct(demandChannel, demandChannel#69875, demandStream, demandStream#69876, considerVariance, considerVariance#69877, demandTimeBuckets, demandTimeBuckets#69878, demandPointInTimeStart, demandPointInTimeStart#69879, demandPointInTimeEnd, demandPointInTimeEnd#69880, demandPointInTime, demandPointInTime#69881) END AS rawDemands#69908]
Input [17]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, product#69873, plant#69874, demandChannel#69875, demandStream#69876, considerVariance#69877, demandTimeBuckets#69878, demandPointInTimeStart#69879, demandPointInTimeEnd#69880, demandPointInTime#69881]
(274) ObjectHashAggregate
Input [9]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, rawDemands#69908]
Keys [9]: [product#69978, plant#69979, orderDateTime#69980, knownfloatingpointnormalized(if (isnull(productPlant#69990)) null else named_struct(systemId, productPlant#69990.systemId, internalUUID, productPlant#69990.internalUUID, plant, productPlant#69990.plant, product, productPlant#69990.product, blockedForReplenishmentStartingFrom, productPlant#69990.blockedForReplenishmentStartingFrom, productionAspect, knownfloatingpointnormalized(if (isnull(productPlant#69990.productionAspect)) null else named_struct(productMovementPlants, productPlant#69990.productionAspect.productMovementPlants, productPlanningPlants, knownfloatingpointnormalized(if (isnull(productPlant#69990.productionAspect.productPlanningPlants)) null else named_struct(mrpType, productPlant#69990.productionAspect.productPlanningPlants.mrpType, abcIndicator, productPlant#69990.productionAspect.productPlanningPlants.abcIndicator, mrpController, productPlant#69990.productionAspect.productPlanningPlants.mrpController, minimumLotSize, knownfloatingpointnormalized(normalizenanandzero(productPlant#69990.productionAspect.productPlanningPlants.minimumLotSize)), minimumSafetyStock, knownfloatingpointnormalized(normalizenanandzero(productPlant#69990.productionAspect.productPlanningPlants.minimumSafetyStock)), logisticalRoundingProfile, productPlant#69990.productionAspect.productPlanningPlants.logisticalRoundingProfile, sourceOfSupplyCategory, productPlant#69990.productionAspect.productPlanningPlants.sourceOfSupplyCategory)))), salesPlant, productPlant#69990.salesPlant, listing, productPlant#69990.listing)) AS productPlant#69990, timezonecode#70005, knownfloatingpointnormalized(normalizenanandzero(outboundUnit#70022)) AS outboundUnit#70022, stockDatetime#70329, knownfloatingpointnormalized(transform(stockBatchQuantities#70330, lambdafunction(knownfloatingpointnormalized(normalizenanandzero(lambda arg#85165)), lambda arg#85165, false))) AS stockBatchQuantities#70330, [7226582400000000] AS [7226582400000000]#85164]
Functions [1]: [partial_collect_list(rawDemands#69908, 0, 0)]
Aggregate Attributes [1]: [buf#73954]
Results [10]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, [7226582400000000]#85164, buf#73955]
(275) ObjectHashAggregate
Input [10]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, [7226582400000000]#85164, buf#73955]
Keys [9]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, [7226582400000000]#85164]
Functions [1]: [collect_list(rawDemands#69908, 0, 0)]
Aggregate Attributes [1]: [collect_list(rawDemands#69908, 0, 0)#69939]
Results [10]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, [7226582400000000] AS stockBatchShelfLifeEndDatetimes#70331, collect_list(rawDemands#69908, 0, 0)#69939 AS rawDemands#69921]
(276) DeserializeToObject [codegen id : 107]
Input [10]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, stockBatchShelfLifeEndDatetimes#70331, rawDemands#69921]
Arguments: createexternalrow(invoke(product#69978.toString()), invoke(plant#69979.toString()), static_invoke(DateTimeUtils.toJavaTimestamp(orderDateTime#69980)), if (isnull(productPlant#69990)) null else createexternalrow(invoke(productPlant#69990.systemId.toString()), invoke(productPlant#69990.internalUUID.toString()), if (isnull(productPlant#69990.plant)) null else createexternalrow(invoke(productPlant#69990.plant.internalRefUUID.toString()), invoke(productPlant#69990.plant.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), if (isnull(productPlant#69990.product)) null else createexternalrow(invoke(productPlant#69990.product.internalRefUUID.toString()), invoke(productPlant#69990.product.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(DateTimeUtils.toJavaDate(productPlant#69990.blockedForReplenishmentStartingFrom)), if (isnull(productPlant#69990.productionAspect)) null else createexternalrow(if (isnull(productPlant#69990.productionAspect.productMovementPlants)) null else createexternalrow(if (isnull(productPlant#69990.productionAspect.productMovementPlants.unitOfIssue)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productMovementPlants.unitOfIssue.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), if (isnull(productPlant#69990.productionAspect.productPlanningPlants)) null else createexternalrow(if (isnull(productPlant#69990.productionAspect.productPlanningPlants.mrpType)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productPlanningPlants.mrpType.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productPlanningPlants.mrpType.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#69990.productionAspect.productPlanningPlants.abcIndicator)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productPlanningPlants.abcIndicator.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productPlanningPlants.abcIndicator.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#69990.productionAspect.productPlanningPlants.mrpController)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productPlanningPlants.mrpController.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productPlanningPlants.mrpController.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(java.lang.Double.valueOf(productPlant#69990.productionAspect.productPlanningPlants.minimumLotSize)), static_invoke(java.lang.Double.valueOf(productPlant#69990.productionAspect.productPlanningPlants.minimumSafetyStock)), if (isnull(productPlant#69990.productionAspect.productPlanningPlants.logisticalRoundingProfile)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productPlanningPlants.logisticalRoundingProfile.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productPlanningPlants.logisticalRoundingProfile.logisticalRoundingProfile.toString()), StructField(internalRefUUID,StringType,true), StructField(logisticalRoundingProfile,StringType,true)), if (isnull(productPlant#69990.productionAspect.productPlanningPlants.sourceOfSupplyCategory)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productPlanningPlants.sourceOfSupplyCategory.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productPlanningPlants.sourceOfSupplyCategory.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(minimumLotSize,DoubleType,true), StructField(minimumSafetyStock,DoubleType,true), StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true), StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)), if (isnull(productPlant#69990.salesPlant)) null else createexternalrow(if (isnull(productPlant#69990.salesPlant.loadingGroup)) null else createexternalrow(invoke(productPlant#69990.salesPlant.loadingGroup.internalRefUUID.toString()), invoke(productPlant#69990.salesPlant.loadingGroup.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), mapobjects(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1), if (isnull(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validFrom)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validTo)), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).isListed)), StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true)), productPlant#69990.listing, Some(class scala.collection.mutable.ArraySeq)), StructField(systemId,StringType,true), StructField(internalUUID,StringType,true), StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(blockedForReplenishmentStartingFrom,DateType,true), StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true), StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)), invoke(timezonecode#70005.toString()), static_invoke(java.lang.Double.valueOf(outboundUnit#70022)), static_invoke(DateTimeUtils.toJavaTimestamp(stockDatetime#70329)), mapobjects(lambdavariable(MapObject, DoubleType, true, -2), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, DoubleType, true, -2))), stockBatchQuantities#70330, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, TimestampType, false, -3), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, TimestampType, false, -3))), stockBatchShelfLifeEndDatetimes#70331, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4), createexternalrow(invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandChannel.toString()), invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandStream.toString()), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).considerVariance)), mapobjects(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5), if (isnull(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandStartDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandVariance)), StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandTimeBuckets, Some(class scala.collection.mutable.ArraySeq)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeStart)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeEnd)), mapobjects(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6), if (isnull(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandVariance)), StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTime, Some(class scala.collection.mutable.ArraySeq)), StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)), rawDemands#69921, Some(class scala.collection.mutable.ArraySeq)), StructField(product,StringType,true), StructField(plant,StringType,true), StructField(orderDateTime,TimestampType,true), StructField(productPlant,StructType(StructField(systemId,StringType,true),StructField(internalUUID,StringType,true),StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(blockedForReplenishmentStartingFrom,DateType,true),StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true),StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)),true), StructField(timezonecode,StringType,true), StructField(outboundUnit,DoubleType,true), StructField(stockDatetime,TimestampType,true), StructField(stockBatchQuantities,ArrayType(DoubleType,true),false), StructField(stockBatchShelfLifeEndDatetimes,ArrayType(TimestampType,false),false), StructField(rawDemands,ArrayType(StructType(StructField(demandChannel,StringType,true),StructField(demandStream,StringType,true),StructField(considerVariance,BooleanType,true),StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true),StructField(demandPointInTimeStart,TimestampType,true),StructField(demandPointInTimeEnd,TimestampType,true),StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)),false),false)), obj#69963: org.apache.spark.sql.Row
(277) MapElements [codegen id : 107]
Input [1]: [obj#69963]
Arguments: com.sap.s4hana.eureka.business.crporderquantitykpiservice.core.business.datasnapshot.DemandTransformatorImpl$$Lambda/0x00000008024b0d88@783ec352, obj#69968: org.apache.spark.sql.Row
(278) SerializeFromObject [codegen id : 107]
Input [1]: [obj#69968]
Arguments: [if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(0))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 0, product), StringType, ObjectType(class java.lang.String)))) AS product#69973, if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(1))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 1, plant), StringType, ObjectType(class java.lang.String)))) AS plant#69974]
(279) Exchange
Input [2]: [product#69973, plant#69974]
Arguments: hashpartitioning(coalesce(plant#69974, ), isnull(plant#69974), coalesce(product#69973, ), isnull(product#69973), 25), ENSURE_REQUIREMENTS, [plan_id=128251]
(280) ShuffleQueryStage
Output [2]: [product#69973, plant#69974]
Arguments: 185
(281) AQEShuffleRead
Input [2]: [product#69973, plant#69974]
Arguments: local
(282) BroadcastExchange
Input [2]: [product#69973, plant#69974]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[1, string, true], ), isnull(input[1, string, true]), coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=128969]
(283) BroadcastQueryStage
Output [2]: [product#69973, plant#69974]
Arguments: 186
(284) BroadcastHashJoin [codegen id : 108]
Left keys [4]: [coalesce(plant#69020, ), isnull(plant#69020), coalesce(product#69019, ), isnull(product#69019)]
Right keys [4]: [coalesce(plant#69974, ), isnull(plant#69974), coalesce(product#69973, ), isnull(product#69973)]
Join type: LeftOuter
Join condition: None
(285) Project [codegen id : 108]
Output [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Input [6]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868, product#69973, plant#69974]
(286) Exchange
Input [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: hashpartitioning(plant#69020, 25), ENSURE_REQUIREMENTS, [plan_id=129040]
(287) ShuffleQueryStage
Output [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: 187
(288) AQEShuffleRead
Input [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: local
(289) BroadcastExchange
Input [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: HashedRelationBroadcastMode(List(input[1, string, true]),false), [plan_id=129090]
(290) BroadcastQueryStage
Output [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: 188
(291) ReusedExchange [Reuses operator id: 55]
Output [1]: [distributionCenters#70815]
(292) ShuffleQueryStage
Output [1]: [distributionCenters#70815]
Arguments: 87
(293) Generate [codegen id : 60]
Input [1]: [distributionCenters#70815]
Arguments: explode(distributionCenters#70815), false, [distributionCenters#70854]
(294) Filter [codegen id : 60]
Input [1]: [distributionCenters#70854]
Condition : (((size(distributionCenters#70854.storageLocations.internalRefUUID, true) > 0) AND isnotnull(distributionCenters#70854.storageLocations.internalRefUUID)) AND isnotnull(distributionCenters#70854.internalRefUUID))
(295) Project [codegen id : 60]
Output [2]: [distributionCenters#70854.internalRefUUID AS _extract_internalRefUUID#71061, distributionCenters#70854.storageLocations.internalRefUUID AS _extract_internalRefUUID#71429]
Input [1]: [distributionCenters#70854]
(296) Generate [codegen id : 60]
Input [2]: [_extract_internalRefUUID#71061, _extract_internalRefUUID#71429]
Arguments: explode(_extract_internalRefUUID#71429), [_extract_internalRefUUID#71061], false, [storageLocations#70855]
(297) Filter [codegen id : 60]
Input [2]: [_extract_internalRefUUID#71061, storageLocations#70855]
Condition : isnotnull(storageLocations#70855)
(298) Project [codegen id : 60]
Output [2]: [_extract_internalRefUUID#71061 AS plantUuid#70858, storageLocations#70855 AS storageLocUuid#70862]
Input [2]: [_extract_internalRefUUID#71061, storageLocations#70855]
(299) BroadcastExchange
Input [2]: [plantUuid#70858, storageLocUuid#70862]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=65566]
(300) BroadcastQueryStage
Output [2]: [plantUuid#70858, storageLocUuid#70862]
Arguments: 104
(301) ReusedExchange [Reuses operator id: 91]
Output [1]: [plant#70866]
(302) ShuffleQueryStage
Output [1]: [plant#70866]
Arguments: 160
(303) HashAggregate
Input [1]: [plant#70866]
Keys [1]: [plant#70866]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70866]
(304) BroadcastHashJoin
Left keys [1]: [plantUuid#70858]
Right keys [1]: [plant#70866]
Join type: Inner
Join condition: None
(305) Project
Output [2]: [plantUuid#70858 AS plant#70807, storageLocUuid#70862]
Input [3]: [plantUuid#70858, storageLocUuid#70862, plant#70866]
(306) BroadcastHashJoin [codegen id : 109]
Left keys [1]: [plant#69020]
Right keys [1]: [plant#70807]
Join type: Inner
Join condition: None
(307) Project [codegen id : 109]
Output [5]: [plant#69020, product#69019, timezonecode#69079, stockDatetime#69868, storageLocUuid#70862]
Input [6]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868, plant#70807, storageLocUuid#70862]
(308) Scan parquet
Output [6]: [internalUUID#70730, id#70731, supplier#70733, isMarkedForDeletion#70736, items#70737, systemId#70729]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-procurement-purchaseorder]
PushedFilters: [IsNotNull(isMarkedForDeletion), Not(EqualTo(isMarkedForDeletion,true)), IsNotNull(items)]
ReadSchema: struct<internalUUID:string,id:string,supplier:struct<internalRefUUID:string,id:string>,isMarkedForDeletion:boolean,items:array<struct<number:string,completeIndicator:boolean,returnIndicator:boolean,deletionIndicator:boolean,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:double,quantityUnit:struct<internalRefUUID:string,code:string>,quantityNumerator:double,quantityDenominator:double,scheduleLines:array<struct<number:string,deliveryAt:timestamp,quantity:double,quantityUnit:struct<internalRefUUID:string,code:string>,receivedQuantity:double,deliveredQuantity:double,productAvailabilityDateTime:timestamp>>,confirmations:array<struct<number:string,confirmationCategory:struct<internalRefUUID:string,code:string>,confirmationDeliveryDate:date,confirmedDeliveryAt:timestamp,confirmedQuantity:double,mrpRelevantQuantity:double,confirmationCreationCategory:string,deletionIndicator:boolean,isRelevantForConfirmation:boolean,delivery:struct<internalRefUUID:string,id:string>,deliveryItemNumber:string>>>>>
(309) ColumnarToRow [codegen id : 55]
Input [6]: [internalUUID#70730, id#70731, supplier#70733, isMarkedForDeletion#70736, items#70737, systemId#70729]
(310) Filter [codegen id : 55]
Input [6]: [internalUUID#70730, id#70731, supplier#70733, isMarkedForDeletion#70736, items#70737, systemId#70729]
Condition : (((isnotnull(isMarkedForDeletion#70736) AND NOT isMarkedForDeletion#70736) AND (size(items#70737, true) > 0)) AND isnotnull(items#70737))
(311) Project [codegen id : 55]
Output [4]: [internalUUID#70730, id#70731, supplier#70733, items#70737]
Input [6]: [internalUUID#70730, id#70731, supplier#70733, isMarkedForDeletion#70736, items#70737, systemId#70729]
(312) Exchange
Input [4]: [internalUUID#70730, id#70731, supplier#70733, items#70737]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=53248]
(313) ShuffleQueryStage
Output [4]: [internalUUID#70730, id#70731, supplier#70733, items#70737]
Arguments: 96
(314) Generate [codegen id : 72]
Input [4]: [internalUUID#70730, id#70731, supplier#70733, items#70737]
Arguments: explode(items#70737), [internalUUID#70730, id#70731, supplier#70733], false, [item#70753]
(315) Filter [codegen id : 72]
Input [4]: [internalUUID#70730, id#70731, supplier#70733, item#70753]
Condition : ((((isnotnull(item#70753.deletionIndicator) AND isnotnull(item#70753.completeIndicator)) AND isnotnull(item#70753.returnIndicator)) AND ((NOT item#70753.deletionIndicator AND NOT item#70753.completeIndicator) AND NOT item#70753.returnIndicator)) AND ((isnotnull(item#70753.product.internalRefUUID) AND isnotnull(item#70753.plant.internalRefUUID)) AND isnotnull(item#70753.storageLocation.internalRefUUID)))
(316) Project [codegen id : 72]
Output [14]: [id#70731, internalUUID#70730, supplier#70733.internalRefUUID AS supplierUuid#70747, item#70753.plant.internalRefUUID AS plant#70770, item#70753.product.internalRefUUID AS product#70773, item#70753.plant.id AS externalPlantId#70775, item#70753.product.id AS externalProductId#70777, item#70753.storageLocation.internalRefUUID AS storageLocUuid#70778, item#70753.quantityUnit.code AS quantityUom#70779, item#70753.quantityUnit.internalRefUUID AS quantityUomUuid#70780, item#70753.quantityNumerator AS quantityNumerator#70781, item#70753.quantityDenominator AS quantityDenominator#70782, item#70753.scheduleLines AS scheduleLines#70783, item#70753.confirmations AS confirmations#70784]
Input [4]: [internalUUID#70730, id#70731, supplier#70733, item#70753]
(317) BroadcastExchange
Input [14]: [id#70731, internalUUID#70730, supplierUuid#70747, plant#70770, product#70773, externalPlantId#70775, externalProductId#70777, storageLocUuid#70778, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, confirmations#70784]
Arguments: HashedRelationBroadcastMode(List(input[4, string, true], input[3, string, true], input[7, string, true]),false), [plan_id=79540]
(318) BroadcastQueryStage
Output [14]: [id#70731, internalUUID#70730, supplierUuid#70747, plant#70770, product#70773, externalPlantId#70775, externalProductId#70777, storageLocUuid#70778, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, confirmations#70784]
Arguments: 142
(319) BroadcastHashJoin [codegen id : 109]
Left keys [3]: [product#69019, plant#69020, storageLocUuid#70862]
Right keys [3]: [product#70773, plant#70770, storageLocUuid#70778]
Join type: Inner
Join condition: None
(320) Project
Output [17]: [id#70731, internalUUID#70730, supplierUuid#70747, plant#70770, product#70773, externalPlantId#70775, externalProductId#70777, storageLocUuid#70778, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, confirmations#70784, stockDatetime#69868, timezonecode#69079, filter(confirmations#70784, lambdafunction(((NOT lambda x_2#70966.deletionIndicator AND lambda x_2#70966.isRelevantForConfirmation) AND isnotnull(lambda x_2#70966.confirmedDeliveryAt)), lambda x_2#70966, false)) AS validConfirmations#70963]
Input [19]: [plant#69020, product#69019, timezonecode#69079, stockDatetime#69868, storageLocUuid#70862, id#70731, internalUUID#70730, supplierUuid#70747, plant#70770, product#70773, externalPlantId#70775, externalProductId#70777, storageLocUuid#70778, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, confirmations#70784]
(321) ResultQueryStage
Output [17]: [id#70731, internalUUID#70730, supplierUuid#70747, plant#70770, product#70773, externalPlantId#70775, externalProductId#70777, storageLocUuid#70778, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, confirmations#70784, stockDatetime#69868, timezonecode#69079, validConfirmations#70963]
Arguments: 189
(322) Filter
Input [3]: [product#69019, plant#69020, orderDateTime#69021]
Condition : (isnotnull(plant#69020) AND isnotnull(product#69019))
(323) Exchange
Input [3]: [product#69019, plant#69020, orderDateTime#69021]
Arguments: hashpartitioning(product#69019, plant#69020, 25), REPARTITION_BY_NUM, [plan_id=31627]
(324) Project
Output [4]: [plant#69029.internalRefUUID AS _extract_internalRefUUID#71360, product#69031.internalRefUUID AS _extract_internalRefUUID#71357, productionAspect#69036.productMovementPlants.unitOfIssue.code AS _extract_code#71361, productionAspect#69036.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71362]
Input [4]: [plant#69029, product#69031, productionAspect#69036, systemId#69025]
(325) Exchange
Input [4]: [_extract_internalRefUUID#71360, _extract_internalRefUUID#71357, _extract_code#71361, _extract_internalRefUUID#71362]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30972]
(326) Project
Output [4]: [_extract_code#71361, _extract_internalRefUUID#71362, _extract_internalRefUUID#71357, _extract_internalRefUUID#71360]
Input [4]: [_extract_internalRefUUID#71360, _extract_internalRefUUID#71357, _extract_code#71361, _extract_internalRefUUID#71362]
(327) BroadcastExchange
Input [4]: [_extract_code#71361, _extract_internalRefUUID#71362, _extract_internalRefUUID#71357, _extract_internalRefUUID#71360]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32024]
(328) BroadcastHashJoin
Left keys [4]: [coalesce(product#69019, ), isnull(product#69019), coalesce(plant#69020, ), isnull(plant#69020)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71357, ), isnull(_extract_internalRefUUID#71357), coalesce(_extract_internalRefUUID#71360, ), isnull(_extract_internalRefUUID#71360)]
Join type: LeftOuter
Join condition: None
(329) Project
Output [5]: [product#69019, plant#69020, orderDateTime#69021, _extract_code#71361, _extract_internalRefUUID#71362]
Input [7]: [product#69019, plant#69020, orderDateTime#69021, _extract_code#71361, _extract_internalRefUUID#71362, _extract_internalRefUUID#71357, _extract_internalRefUUID#71360]
(330) Project
Output [2]: [internalUUID#69062, address#69068.timeZone.code AS timezonecode#69074]
Input [3]: [internalUUID#69062, address#69068, systemId#69061]
(331) Exchange
Input [2]: [internalUUID#69062, timezonecode#69074]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30978]
(332) BroadcastExchange
Input [2]: [internalUUID#69062, timezonecode#69074]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32033]
(333) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69020, ), isnull(plant#69020)]
Right keys [2]: [coalesce(internalUUID#69062, ), isnull(internalUUID#69062)]
Join type: LeftOuter
Join condition: None
(334) Project
Output [6]: [product#69019, plant#69020, orderDateTime#69021, _extract_code#71361, _extract_internalRefUUID#71362, CASE WHEN isnotnull(timezonecode#69074) THEN timezonecode#69074 ELSE UTC END AS timezonecode#69079]
Input [7]: [product#69019, plant#69020, orderDateTime#69021, _extract_code#71361, _extract_internalRefUUID#71362, internalUUID#69062, timezonecode#69074]
(335) Filter
Input [4]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100, systemId#69093]
Condition : isnotnull(internalUUID#69094)
(336) Project
Output [3]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100]
Input [4]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100, systemId#69093]
(337) Exchange
Input [3]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=30984]
(338) BroadcastExchange
Input [3]: [internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32041]
(339) BroadcastHashJoin
Left keys [1]: [product#69019]
Right keys [1]: [internalUUID#69094]
Join type: LeftOuter
Join condition: None
(340) Project
Output [5]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, filter(unitOfMeasureConversions#69100, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71361) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71362)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#69098.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69098.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#69107]
Input [9]: [product#69019, plant#69020, orderDateTime#69021, _extract_code#71361, _extract_internalRefUUID#71362, timezonecode#69079, internalUUID#69094, baseUnitOfMeasure#69098, unitOfMeasureConversions#69100]
(341) Generate
Input [5]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, filteredUnitOfMeasureConversions#69107]
Arguments: explode(filteredUnitOfMeasureConversions#69107.quantityNumerator), [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, filteredUnitOfMeasureConversions#69107], true, [quantityNumerator#69124]
(342) Project
Output [5]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, filteredUnitOfMeasureConversions#69107.quantityDenominator AS _extract_quantityDenominator#70968]
Input [6]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, filteredUnitOfMeasureConversions#69107, quantityNumerator#69124]
(343) Generate
Input [5]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, _extract_quantityDenominator#70968]
Arguments: explode(_extract_quantityDenominator#70968), [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079], true, [quantityDenominator#69129]
(344) Project
Output [4]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079]
Input [5]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, quantityDenominator#69129]
(345) Sort
Input [4]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079]
Arguments: [product#69019 ASC NULLS FIRST, plant#69020 ASC NULLS FIRST], false, 0
(346) Filter
Input [6]: [internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262, systemId#69257]
Condition : (isnotnull(plant#69260.internalRefUUID) AND isnotnull(product#69259.internalRefUUID))
(347) Project
Output [6]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262]
Input [6]: [internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262, systemId#69257]
(348) Exchange
Input [6]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31000]
(349) BroadcastExchange
Input [6]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=32103]
(350) Filter
Input [2]: [marketUnit#69135, distributionCenters#69141]
Condition : (((marketUnit#69135 <=> Daily_grocery) AND (size(distributionCenters#69141, true) > 0)) AND isnotnull(distributionCenters#69141))
(351) Project
Output [1]: [distributionCenters#69141]
Input [2]: [marketUnit#69135, distributionCenters#69141]
(352) Exchange
Input [1]: [distributionCenters#69141]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31011]
(353) Generate
Input [1]: [distributionCenters#69141]
Arguments: explode(distributionCenters#69141), false, [distributionCenters#69183]
(354) Filter
Input [1]: [distributionCenters#69183]
Condition : (((size(distributionCenters#69183.storageLocations, true) > 0) AND isnotnull(distributionCenters#69183.storageLocations)) AND isnotnull(distributionCenters#69183.internalRefUUID))
(355) Project
Output [2]: [distributionCenters#69183.internalRefUUID AS _extract_internalRefUUID#70977, distributionCenters#69183.storageLocations AS _extract_storageLocations#70978]
Input [1]: [distributionCenters#69183]
(356) Generate
Input [2]: [_extract_internalRefUUID#70977, _extract_storageLocations#70978]
Arguments: explode(_extract_storageLocations#70978), [_extract_internalRefUUID#70977], false, [storageLocations#69197]
(357) Project
Output [3]: [_extract_internalRefUUID#70977 AS plantUuid#69186, storageLocations#69197.id AS storageLocId#69199, storageLocations#69197.internalRefUUID AS storageLocUuid#69200]
Input [2]: [_extract_internalRefUUID#70977, storageLocations#69197]
(358) BroadcastExchange
Input [3]: [plantUuid#69186, storageLocId#69199, storageLocUuid#69200]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32097]
(359) Filter
Input [2]: [product#69771, plant#69772]
Condition : isnotnull(plant#69772)
(360) Exchange
Input [2]: [product#69771, plant#69772]
Arguments: hashpartitioning(product#69771, plant#69772, 25), REPARTITION_BY_NUM, [plan_id=31648]
(361) Scan parquet
Output [4]: [plant#69777, product#69778, productionAspect#69780, systemId#69775]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(362) Project
Output [4]: [plant#69777.internalRefUUID AS _extract_internalRefUUID#71368, product#69778.internalRefUUID AS _extract_internalRefUUID#71367, productionAspect#69780.productMovementPlants.unitOfIssue.code AS _extract_code#71369, productionAspect#69780.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71370]
Input [4]: [plant#69777, product#69778, productionAspect#69780, systemId#69775]
(363) Exchange
Input [4]: [_extract_internalRefUUID#71368, _extract_internalRefUUID#71367, _extract_code#71369, _extract_internalRefUUID#71370]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31038]
(364) Project
Output [4]: [_extract_code#71369, _extract_internalRefUUID#71370, _extract_internalRefUUID#71367, _extract_internalRefUUID#71368]
Input [4]: [_extract_internalRefUUID#71368, _extract_internalRefUUID#71367, _extract_code#71369, _extract_internalRefUUID#71370]
(365) BroadcastExchange
Input [4]: [_extract_code#71369, _extract_internalRefUUID#71370, _extract_internalRefUUID#71367, _extract_internalRefUUID#71368]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32061]
(366) BroadcastHashJoin
Left keys [4]: [coalesce(product#69771, ), isnull(product#69771), coalesce(plant#69772, ), isnull(plant#69772)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71367, ), isnull(_extract_internalRefUUID#71367), coalesce(_extract_internalRefUUID#71368, ), isnull(_extract_internalRefUUID#71368)]
Join type: LeftOuter
Join condition: None
(367) Project
Output [4]: [product#69771, plant#69772, _extract_code#71369, _extract_internalRefUUID#71370]
Input [6]: [product#69771, plant#69772, _extract_code#71369, _extract_internalRefUUID#71370, _extract_internalRefUUID#71367, _extract_internalRefUUID#71368]
(368) Project
Output [1]: [internalUUID#69798]
Input [2]: [internalUUID#69798, systemId#69797]
(369) Exchange
Input [1]: [internalUUID#69798]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31049]
(370) BroadcastExchange
Input [1]: [internalUUID#69798]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32070]
(371) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69772, ), isnull(plant#69772)]
Right keys [2]: [coalesce(internalUUID#69798, ), isnull(internalUUID#69798)]
Join type: LeftOuter
Join condition: None
(372) Project
Output [4]: [product#69771, plant#69772, _extract_code#71369, _extract_internalRefUUID#71370]
Input [5]: [product#69771, plant#69772, _extract_code#71369, _extract_internalRefUUID#71370, internalUUID#69798]
(373) Scan parquet
Output [4]: [internalUUID#69826, baseUnitOfMeasure#69830, unitOfMeasureConversions#69832, systemId#69825]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(374) Filter
Input [4]: [internalUUID#69826, baseUnitOfMeasure#69830, unitOfMeasureConversions#69832, systemId#69825]
Condition : isnotnull(internalUUID#69826)
(375) Project
Output [3]: [internalUUID#69826, baseUnitOfMeasure#69830, unitOfMeasureConversions#69832]
Input [4]: [internalUUID#69826, baseUnitOfMeasure#69830, unitOfMeasureConversions#69832, systemId#69825]
(376) Exchange
Input [3]: [internalUUID#69826, baseUnitOfMeasure#69830, unitOfMeasureConversions#69832]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31061]
(377) BroadcastExchange
Input [3]: [internalUUID#69826, baseUnitOfMeasure#69830, unitOfMeasureConversions#69832]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32076]
(378) BroadcastHashJoin
Left keys [1]: [product#69771]
Right keys [1]: [internalUUID#69826]
Join type: LeftOuter
Join condition: None
(379) Project
Output [2]: [plant#69772, filter(unitOfMeasureConversions#69832, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71369) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71370)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#69830.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69830.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#69849]
Input [7]: [product#69771, plant#69772, _extract_code#71369, _extract_internalRefUUID#71370, internalUUID#69826, baseUnitOfMeasure#69830, unitOfMeasureConversions#69832]
(380) Generate
Input [2]: [plant#69772, filteredUnitOfMeasureConversions#69849]
Arguments: explode(filteredUnitOfMeasureConversions#69849.quantityNumerator), [plant#69772, filteredUnitOfMeasureConversions#69849], true, [quantityNumerator#69851]
(381) Project
Output [2]: [plant#69772, filteredUnitOfMeasureConversions#69849.quantityDenominator AS _extract_quantityDenominator#70980]
Input [3]: [plant#69772, filteredUnitOfMeasureConversions#69849, quantityNumerator#69851]
(382) Generate
Input [2]: [plant#69772, _extract_quantityDenominator#70980]
Arguments: explode(_extract_quantityDenominator#70980), [plant#69772], true, [quantityDenominator#69854]
(383) Project
Output [1]: [plant#69772]
Input [2]: [plant#69772, quantityDenominator#69854]
(384) HashAggregate
Input [1]: [plant#69772]
Keys [1]: [plant#69772]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69772]
(385) Exchange
Input [1]: [plant#69772]
Arguments: hashpartitioning(plant#69772, 25), ENSURE_REQUIREMENTS, [plan_id=32092]
(386) HashAggregate
Input [1]: [plant#69772]
Keys [1]: [plant#69772]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69772]
(387) BroadcastHashJoin
Left keys [1]: [plantUuid#69186]
Right keys [1]: [plant#69772]
Join type: Inner
Join condition: None
(388) Project
Output [3]: [plantUuid#69186, storageLocId#69199, storageLocUuid#69200]
Input [4]: [plantUuid#69186, storageLocId#69199, storageLocUuid#69200, plant#69772]
(389) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69260.internalRefUUID, ), isnull(plant#69260.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#69186, ), isnull(plantUuid#69186)]
Join type: Inner
Join condition: None
(390) Generate
Input [9]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, stockQuantities#69262, plantUuid#69186, storageLocId#69199, storageLocUuid#69200]
Arguments: explode(filter(stockQuantities#69262, lambdafunction(((lambda x#69269.storageLocation.internalRefUuid = storageLocUuid#69200) AND (lambda x#69269.storageLocation.Id = storageLocId#69199)), lambda x#69269, false))), [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, plantUuid#69186], false, [stockQuantity#69272]
(391) ObjectHashAggregate
Input [7]: [systemId#69257, internalUUID#69258, product#69259, plant#69260, calculationDateTime#69261, plantUuid#69186, stockQuantity#69272]
Keys [5]: [plantUuid#69186, internalUUID#69258, plant#69260, product#69259, systemId#69257]
Functions [2]: [partial_first(calculationDateTime#69261, false), partial_collect_list(stockQuantity#69272, 0, 0)]
Aggregate Attributes [3]: [first#73924, valueSet#73925, buf#73926]
Results [8]: [plantUuid#69186, internalUUID#69258, plant#69260, product#69259, systemId#69257, first#73927, valueSet#73928, buf#73929]
(392) ObjectHashAggregate
Input [8]: [plantUuid#69186, internalUUID#69258, plant#69260, product#69259, systemId#69257, first#73927, valueSet#73928, buf#73929]
Keys [5]: [plantUuid#69186, internalUUID#69258, plant#69260, product#69259, systemId#69257]
Functions [2]: [first(calculationDateTime#69261, false), collect_list(stockQuantity#69272, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#69261)()#69286, collect_list(stockQuantity#69272, 0, 0)#69287]
Results [4]: [product#69259, plant#69260, first(calculationDateTime#69261)()#69286 AS calculationDateTime#69273, collect_list(stockQuantity#69272, 0, 0)#69287 AS stockQuantities#69274]
(393) Filter
Input [4]: [product#69259, plant#69260, calculationDateTime#69273, stockQuantities#69274]
Condition : ((size(stockQuantities#69274, true) > 0) AND isnotnull(calculationDateTime#69273))
(394) Generate
Input [4]: [product#69259, plant#69260, calculationDateTime#69273, stockQuantities#69274]
Arguments: explode(stockQuantities#69274), [product#69259, plant#69260, calculationDateTime#69273], false, [stockQuantity#69289]
(395) Filter
Input [4]: [product#69259, plant#69260, calculationDateTime#69273, stockQuantity#69289]
Condition : (((isnotnull(stockQuantity#69289.specialStockIndicator.code) AND isnotnull(stockQuantity#69289.stockType.code)) AND (stockQuantity#69289.specialStockIndicator.code = )) AND (stockQuantity#69289.stockType.code = 01))
(396) Project
Output [4]: [plant#69260.internalRefUUID AS plantUuid#69291, product#69259.internalRefUUID AS productUuid#69293, calculationDateTime#69273, stockQuantity#69289.storagelocation.id AS storageLocId#69294]
Input [4]: [product#69259, plant#69260, calculationDateTime#69273, stockQuantity#69289]
(397) Exchange
Input [4]: [plantUuid#69291, productUuid#69293, calculationDateTime#69273, storageLocId#69294]
Arguments: hashpartitioning(productUuid#69293, plantUuid#69291, 25), ENSURE_REQUIREMENTS, [plan_id=32154]
(398) Sort
Input [4]: [plantUuid#69291, productUuid#69293, calculationDateTime#69273, storageLocId#69294]
Arguments: [productUuid#69293 ASC NULLS FIRST, plantUuid#69291 ASC NULLS FIRST], false, 0
(399) Filter
Input [3]: [product#69323, plant#69324, orderDateTime#69325]
Condition : ((isnotnull(orderDateTime#69325) AND isnotnull(plant#69324)) AND isnotnull(product#69323))
(400) Exchange
Input [3]: [product#69323, plant#69324, orderDateTime#69325]
Arguments: hashpartitioning(product#69323, plant#69324, 25), REPARTITION_BY_NUM, [plan_id=31695]
(401) Scan parquet
Output [4]: [plant#69329, product#69330, productionAspect#69332, systemId#69327]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(402) Project
Output [4]: [plant#69329.internalRefUUID AS _extract_internalRefUUID#71377, product#69330.internalRefUUID AS _extract_internalRefUUID#71378, productionAspect#69332.productMovementPlants.unitOfIssue.code AS _extract_code#71375, productionAspect#69332.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71376]
Input [4]: [plant#69329, product#69330, productionAspect#69332, systemId#69327]
(403) Exchange
Input [4]: [_extract_internalRefUUID#71377, _extract_internalRefUUID#71378, _extract_code#71375, _extract_internalRefUUID#71376]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31116]
(404) Project
Output [4]: [_extract_code#71375, _extract_internalRefUUID#71376, _extract_internalRefUUID#71378, _extract_internalRefUUID#71377]
Input [4]: [_extract_internalRefUUID#71377, _extract_internalRefUUID#71378, _extract_code#71375, _extract_internalRefUUID#71376]
(405) BroadcastExchange
Input [4]: [_extract_code#71375, _extract_internalRefUUID#71376, _extract_internalRefUUID#71378, _extract_internalRefUUID#71377]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32123]
(406) BroadcastHashJoin
Left keys [4]: [coalesce(product#69323, ), isnull(product#69323), coalesce(plant#69324, ), isnull(plant#69324)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71378, ), isnull(_extract_internalRefUUID#71378), coalesce(_extract_internalRefUUID#71377, ), isnull(_extract_internalRefUUID#71377)]
Join type: LeftOuter
Join condition: None
(407) Project
Output [5]: [product#69323, plant#69324, orderDateTime#69325, _extract_code#71375, _extract_internalRefUUID#71376]
Input [7]: [product#69323, plant#69324, orderDateTime#69325, _extract_code#71375, _extract_internalRefUUID#71376, _extract_internalRefUUID#71378, _extract_internalRefUUID#71377]
(408) Scan parquet
Output [2]: [internalUUID#69338, systemId#69337]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(409) Project
Output [1]: [internalUUID#69338]
Input [2]: [internalUUID#69338, systemId#69337]
(410) Exchange
Input [1]: [internalUUID#69338]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31129]
(411) BroadcastExchange
Input [1]: [internalUUID#69338]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32127]
(412) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69324, ), isnull(plant#69324)]
Right keys [2]: [coalesce(internalUUID#69338, ), isnull(internalUUID#69338)]
Join type: LeftOuter
Join condition: None
(413) Project
Output [5]: [product#69323, plant#69324, orderDateTime#69325, _extract_code#71375, _extract_internalRefUUID#71376]
Input [6]: [product#69323, plant#69324, orderDateTime#69325, _extract_code#71375, _extract_internalRefUUID#71376, internalUUID#69338]
(414) Scan parquet
Output [4]: [internalUUID#69352, baseUnitOfMeasure#69356, unitOfMeasureConversions#69358, systemId#69351]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(415) Filter
Input [4]: [internalUUID#69352, baseUnitOfMeasure#69356, unitOfMeasureConversions#69358, systemId#69351]
Condition : isnotnull(internalUUID#69352)
(416) Project
Output [3]: [internalUUID#69352, baseUnitOfMeasure#69356, unitOfMeasureConversions#69358]
Input [4]: [internalUUID#69352, baseUnitOfMeasure#69356, unitOfMeasureConversions#69358, systemId#69351]
(417) Exchange
Input [3]: [internalUUID#69352, baseUnitOfMeasure#69356, unitOfMeasureConversions#69358]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31136]
(418) BroadcastExchange
Input [3]: [internalUUID#69352, baseUnitOfMeasure#69356, unitOfMeasureConversions#69358]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32131]
(419) BroadcastHashJoin
Left keys [1]: [product#69323]
Right keys [1]: [internalUUID#69352]
Join type: LeftOuter
Join condition: None
(420) Project
Output [4]: [product#69323, plant#69324, orderDateTime#69325, filter(unitOfMeasureConversions#69358, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71375) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71376)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#69356.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69356.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#69363]
Input [8]: [product#69323, plant#69324, orderDateTime#69325, _extract_code#71375, _extract_internalRefUUID#71376, internalUUID#69352, baseUnitOfMeasure#69356, unitOfMeasureConversions#69358]
(421) Generate
Input [4]: [product#69323, plant#69324, orderDateTime#69325, filteredUnitOfMeasureConversions#69363]
Arguments: explode(filteredUnitOfMeasureConversions#69363.quantityNumerator), [product#69323, plant#69324, orderDateTime#69325, filteredUnitOfMeasureConversions#69363], true, [quantityNumerator#69364]
(422) Project
Output [4]: [product#69323, plant#69324, orderDateTime#69325, filteredUnitOfMeasureConversions#69363.quantityDenominator AS _extract_quantityDenominator#70990]
Input [5]: [product#69323, plant#69324, orderDateTime#69325, filteredUnitOfMeasureConversions#69363, quantityNumerator#69364]
(423) Generate
Input [4]: [product#69323, plant#69324, orderDateTime#69325, _extract_quantityDenominator#70990]
Arguments: explode(_extract_quantityDenominator#70990), [product#69323, plant#69324, orderDateTime#69325], true, [quantityDenominator#69365]
(424) Project
Output [3]: [product#69323, plant#69324, orderDateTime#69325]
Input [4]: [product#69323, plant#69324, orderDateTime#69325, quantityDenominator#69365]
(425) Sort
Input [3]: [product#69323, plant#69324, orderDateTime#69325]
Arguments: [product#69323 ASC NULLS FIRST, plant#69324 ASC NULLS FIRST], false, 0
(426) SortMergeJoin
Left keys [2]: [productUuid#69293, plantUuid#69291]
Right keys [2]: [product#69323, plant#69324]
Join type: Inner
Join condition: (calculationDateTime#69273 <= orderDateTime#69325)
(427) Project
Output [4]: [plantUuid#69291 AS plant#69435, productUuid#69293 AS product#69436, storageLocId#69294, calculationDateTime#69273]
Input [7]: [plantUuid#69291, productUuid#69293, calculationDateTime#69273, storageLocId#69294, product#69323, plant#69324, orderDateTime#69325]
(428) Exchange
Input [4]: [plant#69435, product#69436, storageLocId#69294, calculationDateTime#69273]
Arguments: hashpartitioning(coalesce(plant#69435, ), isnull(plant#69435), coalesce(product#69436, ), isnull(product#69436), coalesce(storageLocId#69294, ), isnull(storageLocId#69294), coalesce(calculationDateTime#69273, 1970-01-01 00:00:00), isnull(calculationDateTime#69273), 25), ENSURE_REQUIREMENTS, [plan_id=32283]
(429) Sort
Input [4]: [plant#69435, product#69436, storageLocId#69294, calculationDateTime#69273]
Arguments: [coalesce(plant#69435, ) ASC NULLS FIRST, isnull(plant#69435) ASC NULLS FIRST, coalesce(product#69436, ) ASC NULLS FIRST, isnull(product#69436) ASC NULLS FIRST, coalesce(storageLocId#69294, ) ASC NULLS FIRST, isnull(storageLocId#69294) ASC NULLS FIRST, coalesce(calculationDateTime#69273, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(calculationDateTime#69273) ASC NULLS FIRST], false, 0
(430) Scan parquet
Output [6]: [internalUUID#69458, product#69459, plant#69460, calculationDateTime#69461, stockQuantities#69462, systemId#69457]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(431) Filter
Input [6]: [internalUUID#69458, product#69459, plant#69460, calculationDateTime#69461, stockQuantities#69462, systemId#69457]
Condition : (isnotnull(plant#69460.internalRefUUID) AND isnotnull(product#69459.internalRefUUID))
(432) Project
Output [6]: [systemId#69457, internalUUID#69458, product#69459, plant#69460, calculationDateTime#69461, stockQuantities#69462]
Input [6]: [internalUUID#69458, product#69459, plant#69460, calculationDateTime#69461, stockQuantities#69462, systemId#69457]
(433) Exchange
Input [6]: [systemId#69457, internalUUID#69458, product#69459, plant#69460, calculationDateTime#69461, stockQuantities#69462]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31161]
(434) BroadcastExchange
Input [6]: [systemId#69457, internalUUID#69458, product#69459, plant#69460, calculationDateTime#69461, stockQuantities#69462]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=32212]
(435) Scan parquet
Output [2]: [marketUnit#69463, distributionCenters#69469]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(436) Filter
Input [2]: [marketUnit#69463, distributionCenters#69469]
Condition : (((marketUnit#69463 <=> Daily_grocery) AND (size(distributionCenters#69469, true) > 0)) AND isnotnull(distributionCenters#69469))
(437) Project
Output [1]: [distributionCenters#69469]
Input [2]: [marketUnit#69463, distributionCenters#69469]
(438) Exchange
Input [1]: [distributionCenters#69469]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31167]
(439) Generate
Input [1]: [distributionCenters#69469]
Arguments: explode(distributionCenters#69469), false, [distributionCenters#69485]
(440) Filter
Input [1]: [distributionCenters#69485]
Condition : (((size(distributionCenters#69485.storageLocations, true) > 0) AND isnotnull(distributionCenters#69485.storageLocations)) AND isnotnull(distributionCenters#69485.internalRefUUID))
(441) Project
Output [2]: [distributionCenters#69485.internalRefUUID AS _extract_internalRefUUID#70997, distributionCenters#69485.storageLocations AS _extract_storageLocations#70998]
Input [1]: [distributionCenters#69485]
(442) Generate
Input [2]: [_extract_internalRefUUID#70997, _extract_storageLocations#70998]
Arguments: explode(_extract_storageLocations#70998), [_extract_internalRefUUID#70997], false, [storageLocations#69486]
(443) Project
Output [3]: [_extract_internalRefUUID#70997 AS plantUuid#69488, storageLocations#69486.id AS storageLocId#69489, storageLocations#69486.internalRefUUID AS storageLocUuid#69490]
Input [2]: [_extract_internalRefUUID#70997, storageLocations#69486]
(444) BroadcastExchange
Input [3]: [plantUuid#69488, storageLocId#69489, storageLocUuid#69490]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32204]
(445) Scan parquet
Output [2]: [product#69491, plant#69492]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(446) Filter
Input [2]: [product#69491, plant#69492]
Condition : isnotnull(plant#69492)
(447) Exchange
Input [2]: [product#69491, plant#69492]
Arguments: hashpartitioning(product#69491, plant#69492, 25), REPARTITION_BY_NUM, [plan_id=31720]
(448) Scan parquet
Output [4]: [plant#69497, product#69498, productionAspect#69500, systemId#69495]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(449) Project
Output [4]: [plant#69497.internalRefUUID AS _extract_internalRefUUID#71390, product#69498.internalRefUUID AS _extract_internalRefUUID#71389, productionAspect#69500.productMovementPlants.unitOfIssue.code AS _extract_code#71387, productionAspect#69500.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71388]
Input [4]: [plant#69497, product#69498, productionAspect#69500, systemId#69495]
(450) Exchange
Input [4]: [_extract_internalRefUUID#71390, _extract_internalRefUUID#71389, _extract_code#71387, _extract_internalRefUUID#71388]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31196]
(451) Project
Output [4]: [_extract_code#71387, _extract_internalRefUUID#71388, _extract_internalRefUUID#71389, _extract_internalRefUUID#71390]
Input [4]: [_extract_internalRefUUID#71390, _extract_internalRefUUID#71389, _extract_code#71387, _extract_internalRefUUID#71388]
(452) BroadcastExchange
Input [4]: [_extract_code#71387, _extract_internalRefUUID#71388, _extract_internalRefUUID#71389, _extract_internalRefUUID#71390]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32164]
(453) BroadcastHashJoin
Left keys [4]: [coalesce(product#69491, ), isnull(product#69491), coalesce(plant#69492, ), isnull(plant#69492)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71389, ), isnull(_extract_internalRefUUID#71389), coalesce(_extract_internalRefUUID#71390, ), isnull(_extract_internalRefUUID#71390)]
Join type: LeftOuter
Join condition: None
(454) Project
Output [4]: [product#69491, plant#69492, _extract_code#71387, _extract_internalRefUUID#71388]
Input [6]: [product#69491, plant#69492, _extract_code#71387, _extract_internalRefUUID#71388, _extract_internalRefUUID#71389, _extract_internalRefUUID#71390]
(455) Scan parquet
Output [2]: [internalUUID#69505, systemId#69504]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(456) Project
Output [1]: [internalUUID#69505]
Input [2]: [internalUUID#69505, systemId#69504]
(457) Exchange
Input [1]: [internalUUID#69505]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31209]
(458) BroadcastExchange
Input [1]: [internalUUID#69505]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32172]
(459) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69492, ), isnull(plant#69492)]
Right keys [2]: [coalesce(internalUUID#69505, ), isnull(internalUUID#69505)]
Join type: LeftOuter
Join condition: None
(460) Project
Output [4]: [product#69491, plant#69492, _extract_code#71387, _extract_internalRefUUID#71388]
Input [5]: [product#69491, plant#69492, _extract_code#71387, _extract_internalRefUUID#71388, internalUUID#69505]
(461) Scan parquet
Output [4]: [internalUUID#69519, baseUnitOfMeasure#69523, unitOfMeasureConversions#69525, systemId#69518]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(462) Filter
Input [4]: [internalUUID#69519, baseUnitOfMeasure#69523, unitOfMeasureConversions#69525, systemId#69518]
Condition : isnotnull(internalUUID#69519)
(463) Project
Output [3]: [internalUUID#69519, baseUnitOfMeasure#69523, unitOfMeasureConversions#69525]
Input [4]: [internalUUID#69519, baseUnitOfMeasure#69523, unitOfMeasureConversions#69525, systemId#69518]
(464) Exchange
Input [3]: [internalUUID#69519, baseUnitOfMeasure#69523, unitOfMeasureConversions#69525]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31218]
(465) BroadcastExchange
Input [3]: [internalUUID#69519, baseUnitOfMeasure#69523, unitOfMeasureConversions#69525]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32180]
(466) BroadcastHashJoin
Left keys [1]: [product#69491]
Right keys [1]: [internalUUID#69519]
Join type: LeftOuter
Join condition: None
(467) Project
Output [2]: [plant#69492, filter(unitOfMeasureConversions#69525, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71387) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71388)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#69523.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69523.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#69530]
Input [7]: [product#69491, plant#69492, _extract_code#71387, _extract_internalRefUUID#71388, internalUUID#69519, baseUnitOfMeasure#69523, unitOfMeasureConversions#69525]
(468) Generate
Input [2]: [plant#69492, filteredUnitOfMeasureConversions#69530]
Arguments: explode(filteredUnitOfMeasureConversions#69530.quantityNumerator), [plant#69492, filteredUnitOfMeasureConversions#69530], true, [quantityNumerator#69531]
(469) Project
Output [2]: [plant#69492, filteredUnitOfMeasureConversions#69530.quantityDenominator AS _extract_quantityDenominator#71002]
Input [3]: [plant#69492, filteredUnitOfMeasureConversions#69530, quantityNumerator#69531]
(470) Generate
Input [2]: [plant#69492, _extract_quantityDenominator#71002]
Arguments: explode(_extract_quantityDenominator#71002), [plant#69492], true, [quantityDenominator#69532]
(471) Project
Output [1]: [plant#69492]
Input [2]: [plant#69492, quantityDenominator#69532]
(472) HashAggregate
Input [1]: [plant#69492]
Keys [1]: [plant#69492]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69492]
(473) Exchange
Input [1]: [plant#69492]
Arguments: hashpartitioning(plant#69492, 25), ENSURE_REQUIREMENTS, [plan_id=32198]
(474) HashAggregate
Input [1]: [plant#69492]
Keys [1]: [plant#69492]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#69492]
(475) BroadcastHashJoin
Left keys [1]: [plantUuid#69488]
Right keys [1]: [plant#69492]
Join type: Inner
Join condition: None
(476) Project
Output [3]: [plantUuid#69488, storageLocId#69489, storageLocUuid#69490]
Input [4]: [plantUuid#69488, storageLocId#69489, storageLocUuid#69490, plant#69492]
(477) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69460.internalRefUUID, ), isnull(plant#69460.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#69488, ), isnull(plantUuid#69488)]
Join type: Inner
Join condition: None
(478) Generate
Input [9]: [systemId#69457, internalUUID#69458, product#69459, plant#69460, calculationDateTime#69461, stockQuantities#69462, plantUuid#69488, storageLocId#69489, storageLocUuid#69490]
Arguments: explode(filter(stockQuantities#69462, lambdafunction(((lambda x#69269.storageLocation.internalRefUuid = storageLocUuid#69490) AND (lambda x#69269.storageLocation.Id = storageLocId#69489)), lambda x#69269, false))), [systemId#69457, internalUUID#69458, product#69459, plant#69460, calculationDateTime#69461, plantUuid#69488], false, [stockQuantity#69535]
(479) ObjectHashAggregate
Input [7]: [systemId#69457, internalUUID#69458, product#69459, plant#69460, calculationDateTime#69461, plantUuid#69488, stockQuantity#69535]
Keys [5]: [plantUuid#69488, internalUUID#69458, plant#69460, product#69459, systemId#69457]
Functions [2]: [partial_first(calculationDateTime#69461, false), partial_collect_list(stockQuantity#69535, 0, 0)]
Aggregate Attributes [3]: [first#73934, valueSet#73935, buf#73936]
Results [8]: [plantUuid#69488, internalUUID#69458, plant#69460, product#69459, systemId#69457, first#73937, valueSet#73938, buf#73939]
(480) ObjectHashAggregate
Input [8]: [plantUuid#69488, internalUUID#69458, plant#69460, product#69459, systemId#69457, first#73937, valueSet#73938, buf#73939]
Keys [5]: [plantUuid#69488, internalUUID#69458, plant#69460, product#69459, systemId#69457]
Functions [2]: [first(calculationDateTime#69461, false), collect_list(stockQuantity#69535, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#69461)()#69286, collect_list(stockQuantity#69535, 0, 0)#69287]
Results [4]: [product#69459, plant#69460, first(calculationDateTime#69461)()#69286 AS calculationDateTime#69536, collect_list(stockQuantity#69535, 0, 0)#69287 AS stockQuantities#69537]
(481) Filter
Input [4]: [product#69459, plant#69460, calculationDateTime#69536, stockQuantities#69537]
Condition : ((size(stockQuantities#69537, true) > 0) AND isnotnull(calculationDateTime#69536))
(482) Generate
Input [4]: [product#69459, plant#69460, calculationDateTime#69536, stockQuantities#69537]
Arguments: explode(stockQuantities#69537), [product#69459, plant#69460, calculationDateTime#69536], false, [stockQuantity#69538]
(483) Filter
Input [4]: [product#69459, plant#69460, calculationDateTime#69536, stockQuantity#69538]
Condition : (((isnotnull(stockQuantity#69538.specialStockIndicator.code) AND isnotnull(stockQuantity#69538.stockType.code)) AND (stockQuantity#69538.specialStockIndicator.code = )) AND (stockQuantity#69538.stockType.code = 01))
(484) Project
Output [4]: [plant#69460.internalRefUUID AS plantUuid#69540, product#69459.internalRefUUID AS productUuid#69542, calculationDateTime#69536, stockQuantity#69538.storagelocation.id AS storageLocId#69543]
Input [4]: [product#69459, plant#69460, calculationDateTime#69536, stockQuantity#69538]
(485) Exchange
Input [4]: [plantUuid#69540, productUuid#69542, calculationDateTime#69536, storageLocId#69543]
Arguments: hashpartitioning(productUuid#69542, plantUuid#69540, 25), ENSURE_REQUIREMENTS, [plan_id=32264]
(486) Sort
Input [4]: [plantUuid#69540, productUuid#69542, calculationDateTime#69536, storageLocId#69543]
Arguments: [productUuid#69542 ASC NULLS FIRST, plantUuid#69540 ASC NULLS FIRST], false, 0
(487) Scan parquet
Output [3]: [product#69548, plant#69549, orderDateTime#69550]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(488) Filter
Input [3]: [product#69548, plant#69549, orderDateTime#69550]
Condition : ((isnotnull(orderDateTime#69550) AND isnotnull(plant#69549)) AND isnotnull(product#69548))
(489) Exchange
Input [3]: [product#69548, plant#69549, orderDateTime#69550]
Arguments: hashpartitioning(product#69548, plant#69549, 25), REPARTITION_BY_NUM, [plan_id=31767]
(490) Scan parquet
Output [4]: [plant#69554, product#69555, productionAspect#69557, systemId#69552]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(491) Project
Output [4]: [plant#69554.internalRefUUID AS _extract_internalRefUUID#71392, product#69555.internalRefUUID AS _extract_internalRefUUID#71391, productionAspect#69557.productMovementPlants.unitOfIssue.code AS _extract_code#71393, productionAspect#69557.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71394]
Input [4]: [plant#69554, product#69555, productionAspect#69557, systemId#69552]
(492) Exchange
Input [4]: [_extract_internalRefUUID#71392, _extract_internalRefUUID#71391, _extract_code#71393, _extract_internalRefUUID#71394]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31272]
(493) Project
Output [4]: [_extract_code#71393, _extract_internalRefUUID#71394, _extract_internalRefUUID#71391, _extract_internalRefUUID#71392]
Input [4]: [_extract_internalRefUUID#71392, _extract_internalRefUUID#71391, _extract_code#71393, _extract_internalRefUUID#71394]
(494) BroadcastExchange
Input [4]: [_extract_code#71393, _extract_internalRefUUID#71394, _extract_internalRefUUID#71391, _extract_internalRefUUID#71392]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32231]
(495) BroadcastHashJoin
Left keys [4]: [coalesce(product#69548, ), isnull(product#69548), coalesce(plant#69549, ), isnull(plant#69549)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71391, ), isnull(_extract_internalRefUUID#71391), coalesce(_extract_internalRefUUID#71392, ), isnull(_extract_internalRefUUID#71392)]
Join type: LeftOuter
Join condition: None
(496) Project
Output [5]: [product#69548, plant#69549, orderDateTime#69550, _extract_code#71393, _extract_internalRefUUID#71394]
Input [7]: [product#69548, plant#69549, orderDateTime#69550, _extract_code#71393, _extract_internalRefUUID#71394, _extract_internalRefUUID#71391, _extract_internalRefUUID#71392]
(497) Scan parquet
Output [2]: [internalUUID#69562, systemId#69561]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(498) Project
Output [1]: [internalUUID#69562]
Input [2]: [internalUUID#69562, systemId#69561]
(499) Exchange
Input [1]: [internalUUID#69562]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31283]
(500) BroadcastExchange
Input [1]: [internalUUID#69562]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32239]
(501) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69549, ), isnull(plant#69549)]
Right keys [2]: [coalesce(internalUUID#69562, ), isnull(internalUUID#69562)]
Join type: LeftOuter
Join condition: None
(502) Project
Output [5]: [product#69548, plant#69549, orderDateTime#69550, _extract_code#71393, _extract_internalRefUUID#71394]
Input [6]: [product#69548, plant#69549, orderDateTime#69550, _extract_code#71393, _extract_internalRefUUID#71394, internalUUID#69562]
(503) Scan parquet
Output [4]: [internalUUID#69576, baseUnitOfMeasure#69580, unitOfMeasureConversions#69582, systemId#69575]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(504) Filter
Input [4]: [internalUUID#69576, baseUnitOfMeasure#69580, unitOfMeasureConversions#69582, systemId#69575]
Condition : isnotnull(internalUUID#69576)
(505) Project
Output [3]: [internalUUID#69576, baseUnitOfMeasure#69580, unitOfMeasureConversions#69582]
Input [4]: [internalUUID#69576, baseUnitOfMeasure#69580, unitOfMeasureConversions#69582, systemId#69575]
(506) Exchange
Input [3]: [internalUUID#69576, baseUnitOfMeasure#69580, unitOfMeasureConversions#69582]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31289]
(507) BroadcastExchange
Input [3]: [internalUUID#69576, baseUnitOfMeasure#69580, unitOfMeasureConversions#69582]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32246]
(508) BroadcastHashJoin
Left keys [1]: [product#69548]
Right keys [1]: [internalUUID#69576]
Join type: LeftOuter
Join condition: None
(509) Project
Output [4]: [product#69548, plant#69549, orderDateTime#69550, filter(unitOfMeasureConversions#69582, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71393) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71394)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#69580.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#69580.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#69587]
Input [8]: [product#69548, plant#69549, orderDateTime#69550, _extract_code#71393, _extract_internalRefUUID#71394, internalUUID#69576, baseUnitOfMeasure#69580, unitOfMeasureConversions#69582]
(510) Generate
Input [4]: [product#69548, plant#69549, orderDateTime#69550, filteredUnitOfMeasureConversions#69587]
Arguments: explode(filteredUnitOfMeasureConversions#69587.quantityNumerator), [product#69548, plant#69549, orderDateTime#69550, filteredUnitOfMeasureConversions#69587], true, [quantityNumerator#69588]
(511) Project
Output [4]: [product#69548, plant#69549, orderDateTime#69550, filteredUnitOfMeasureConversions#69587.quantityDenominator AS _extract_quantityDenominator#71011]
Input [5]: [product#69548, plant#69549, orderDateTime#69550, filteredUnitOfMeasureConversions#69587, quantityNumerator#69588]
(512) Generate
Input [4]: [product#69548, plant#69549, orderDateTime#69550, _extract_quantityDenominator#71011]
Arguments: explode(_extract_quantityDenominator#71011), [product#69548, plant#69549, orderDateTime#69550], true, [quantityDenominator#69589]
(513) Project
Output [3]: [product#69548, plant#69549, orderDateTime#69550]
Input [4]: [product#69548, plant#69549, orderDateTime#69550, quantityDenominator#69589]
(514) Sort
Input [3]: [product#69548, plant#69549, orderDateTime#69550]
Arguments: [product#69548 ASC NULLS FIRST, plant#69549 ASC NULLS FIRST], false, 0
(515) SortMergeJoin
Left keys [2]: [productUuid#69542, plantUuid#69540]
Right keys [2]: [product#69548, plant#69549]
Join type: Inner
Join condition: (calculationDateTime#69536 <= orderDateTime#69550)
(516) Project
Output [4]: [plantUuid#69540 AS plant#69620, productUuid#69542 AS product#69621, storageLocId#69543, calculationDateTime#69536]
Input [7]: [plantUuid#69540, productUuid#69542, calculationDateTime#69536, storageLocId#69543, product#69548, plant#69549, orderDateTime#69550]
(517) HashAggregate
Input [4]: [plant#69620, product#69621, storageLocId#69543, calculationDateTime#69536]
Keys [3]: [plant#69620, product#69621, storageLocId#69543]
Functions [1]: [partial_max(calculationDateTime#69536)]
Aggregate Attributes [1]: [max#73930]
Results [4]: [plant#69620, product#69621, storageLocId#69543, max#73931]
(518) HashAggregate
Input [4]: [plant#69620, product#69621, storageLocId#69543, max#73931]
Keys [3]: [plant#69620, product#69621, storageLocId#69543]
Functions [1]: [max(calculationDateTime#69536)]
Aggregate Attributes [1]: [max(calculationDateTime#69536)#69447]
Results [4]: [plant#69620, product#69621, storageLocId#69543, max(calculationDateTime#69536)#69447 AS max_calc_datetime#69441]
(519) Exchange
Input [4]: [plant#69620, product#69621, storageLocId#69543, max_calc_datetime#69441]
Arguments: hashpartitioning(coalesce(plant#69620, ), isnull(plant#69620), coalesce(product#69621, ), isnull(product#69621), coalesce(storageLocId#69543, ), isnull(storageLocId#69543), coalesce(max_calc_datetime#69441, 1970-01-01 00:00:00), isnull(max_calc_datetime#69441), 25), ENSURE_REQUIREMENTS, [plan_id=32285]
(520) Sort
Input [4]: [plant#69620, product#69621, storageLocId#69543, max_calc_datetime#69441]
Arguments: [coalesce(plant#69620, ) ASC NULLS FIRST, isnull(plant#69620) ASC NULLS FIRST, coalesce(product#69621, ) ASC NULLS FIRST, isnull(product#69621) ASC NULLS FIRST, coalesce(storageLocId#69543, ) ASC NULLS FIRST, isnull(storageLocId#69543) ASC NULLS FIRST, coalesce(max_calc_datetime#69441, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(max_calc_datetime#69441) ASC NULLS FIRST], false, 0
(521) SortMergeJoin
Left keys [8]: [coalesce(plant#69435, ), isnull(plant#69435), coalesce(product#69436, ), isnull(product#69436), coalesce(storageLocId#69294, ), isnull(storageLocId#69294), coalesce(calculationDateTime#69273, 1970-01-01 00:00:00), isnull(calculationDateTime#69273)]
Right keys [8]: [coalesce(plant#69620, ), isnull(plant#69620), coalesce(product#69621, ), isnull(product#69621), coalesce(storageLocId#69543, ), isnull(storageLocId#69543), coalesce(max_calc_datetime#69441, 1970-01-01 00:00:00), isnull(max_calc_datetime#69441)]
Join type: Inner
Join condition: None
(522) Project
Output [3]: [plant#69435, product#69436, calculationDateTime#69273]
Input [8]: [plant#69435, product#69436, storageLocId#69294, calculationDateTime#69273, plant#69620, product#69621, storageLocId#69543, max_calc_datetime#69441]
(523) HashAggregate
Input [3]: [plant#69435, product#69436, calculationDateTime#69273]
Keys [2]: [product#69436, plant#69435]
Functions [1]: [partial_min(calculationDateTime#69273)]
Aggregate Attributes [1]: [min#73914]
Results [3]: [product#69436, plant#69435, min#73915]
(524) Exchange
Input [3]: [product#69436, plant#69435, min#73915]
Arguments: hashpartitioning(product#69436, plant#69435, 25), ENSURE_REQUIREMENTS, [plan_id=32300]
(525) HashAggregate
Input [3]: [product#69436, plant#69435, min#73915]
Keys [2]: [product#69436, plant#69435]
Functions [1]: [min(calculationDateTime#69273)]
Aggregate Attributes [1]: [min(calculationDateTime#69273)#69766]
Results [3]: [product#69436, plant#69435, min(calculationDateTime#69273)#69766 AS minCalculationDateTime#69752]
(526) Sort
Input [3]: [product#69436, plant#69435, minCalculationDateTime#69752]
Arguments: [product#69436 ASC NULLS FIRST, plant#69435 ASC NULLS FIRST], false, 0
(527) SortMergeJoin
Left keys [2]: [product#69019, plant#69020]
Right keys [2]: [product#69436, plant#69435]
Join type: LeftOuter
Join condition: None
(528) Project
Output [4]: [product#69019, plant#69020, timezonecode#69079, CASE WHEN isnotnull(minCalculationDateTime#69752) THEN minCalculationDateTime#69752 ELSE orderDateTime#69021 END AS stockDatetime#69868]
Input [7]: [product#69019, plant#69020, orderDateTime#69021, timezonecode#69079, product#69436, plant#69435, minCalculationDateTime#69752]
(529) Exchange
Input [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: hashpartitioning(coalesce(plant#69020, ), isnull(plant#69020), coalesce(product#69019, ), isnull(product#69019), 25), ENSURE_REQUIREMENTS, [plan_id=32632]
(530) Sort
Input [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: [coalesce(plant#69020, ) ASC NULLS FIRST, isnull(plant#69020) ASC NULLS FIRST, coalesce(product#69019, ) ASC NULLS FIRST, isnull(product#69019) ASC NULLS FIRST], false, 0
(531) Exchange
Input [3]: [product#69978, plant#69979, orderDateTime#69980]
Arguments: hashpartitioning(product#69978, plant#69979, 25), REPARTITION_BY_NUM, [plan_id=31319]
(532) Project
Output [8]: [systemId#69982, internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989]
Input [8]: [internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989, systemId#69982]
(533) Exchange
Input [8]: [systemId#69982, internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31325]
(534) Project
Output [1]: [struct(systemId, systemId#69982, internalUUID, internalUUID#69983, plant, plant#69984, product, product#69985, blockedForReplenishmentStartingFrom, blockedForReplenishmentStartingFrom#69986, productionAspect, productionAspect#69987, salesPlant, salesPlant#69988, listing, listing#69989) AS productPlant#69990]
Input [8]: [systemId#69982, internalUUID#69983, plant#69984, product#69985, blockedForReplenishmentStartingFrom#69986, productionAspect#69987, salesPlant#69988, listing#69989]
(535) BroadcastExchange
Input [1]: [productPlant#69990]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].product.internalRefUUID), coalesce(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID, ), isnull(input[0, struct<systemId:string,internalUUID:string,plant:struct<internalRefUUID:string,id:string>,product:struct<internalRefUUID:string,id:string>,blockedForReplenishmentStartingFrom:date,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>,productPlanningPlants:struct<mrpType:struct<internalRefUUID:string,code:string>,abcIndicator:struct<internalRefUUID:string,code:string>,mrpController:struct<internalRefUUID:string,id:string>,minimumLotSize:double,minimumSafetyStock:double,logisticalRoundingProfile:struct<internalRefUUID:string,logisticalRoundingProfile:string>,sourceOfSupplyCategory:struct<internalRefUUID:string,code:string>>>,salesPlant:struct<loadingGroup:struct<internalRefUUID:string,code:string>>,listing:array<struct<validFrom:timestamp,validTo:timestamp,isListed:boolean>>>, false].plant.internalRefUUID)),false), [plan_id=32316]
(536) BroadcastHashJoin
Left keys [4]: [coalesce(product#69978, ), isnull(product#69978), coalesce(plant#69979, ), isnull(plant#69979)]
Right keys [4]: [coalesce(productPlant#69990.product.internalRefUUID, ), isnull(productPlant#69990.product.internalRefUUID), coalesce(productPlant#69990.plant.internalRefUUID, ), isnull(productPlant#69990.plant.internalRefUUID)]
Join type: LeftOuter
Join condition: None
(537) Scan parquet
Output [3]: [internalUUID#69992, address#69998, systemId#69991]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string,address:struct<timeZone:struct<code:string>>>
(538) Project
Output [2]: [internalUUID#69992, address#69998.timeZone.code AS timezonecode#70003]
Input [3]: [internalUUID#69992, address#69998, systemId#69991]
(539) Exchange
Input [2]: [internalUUID#69992, timezonecode#70003]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31333]
(540) BroadcastExchange
Input [2]: [internalUUID#69992, timezonecode#70003]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32320]
(541) BroadcastHashJoin
Left keys [2]: [coalesce(plant#69979, ), isnull(plant#69979)]
Right keys [2]: [coalesce(internalUUID#69992, ), isnull(internalUUID#69992)]
Join type: LeftOuter
Join condition: None
(542) Project
Output [5]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, CASE WHEN isnotnull(timezonecode#70003) THEN timezonecode#70003 ELSE UTC END AS timezonecode#70005]
Input [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, internalUUID#69992, timezonecode#70003]
(543) Scan parquet
Output [4]: [internalUUID#70007, baseUnitOfMeasure#70011, unitOfMeasureConversions#70013, systemId#70006]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(544) Filter
Input [4]: [internalUUID#70007, baseUnitOfMeasure#70011, unitOfMeasureConversions#70013, systemId#70006]
Condition : isnotnull(internalUUID#70007)
(545) Project
Output [3]: [internalUUID#70007, baseUnitOfMeasure#70011, unitOfMeasureConversions#70013]
Input [4]: [internalUUID#70007, baseUnitOfMeasure#70011, unitOfMeasureConversions#70013, systemId#70006]
(546) Exchange
Input [3]: [internalUUID#70007, baseUnitOfMeasure#70011, unitOfMeasureConversions#70013]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31342]
(547) BroadcastExchange
Input [3]: [internalUUID#70007, baseUnitOfMeasure#70011, unitOfMeasureConversions#70013]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32324]
(548) BroadcastHashJoin
Left keys [1]: [product#69978]
Right keys [1]: [internalUUID#70007]
Join type: LeftOuter
Join condition: None
(549) Project
Output [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, filter(unitOfMeasureConversions#70013, lambdafunction((((lambda u#69109.measurementUnit1.code = productPlant#69990.productionAspect.productMovementPlants.unitOfIssue.code) AND (lambda u#69109.measurementUnit1.internalRefUUID = productPlant#69990.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#70011.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70011.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#70018]
Input [8]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, internalUUID#70007, baseUnitOfMeasure#70011, unitOfMeasureConversions#70013]
(550) Project
Output [7]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, filteredUnitOfMeasureConversions#70018.quantityDenominator AS _extract_quantityDenominator#71133, filteredUnitOfMeasureConversions#70018.quantityNumerator AS _extract_quantityNumerator#71134]
Input [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, filteredUnitOfMeasureConversions#70018]
(551) Generate
Input [7]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, _extract_quantityDenominator#71133, _extract_quantityNumerator#71134]
Arguments: explode(_extract_quantityNumerator#71134), [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, _extract_quantityDenominator#71133], true, [quantityNumerator#70019]
(552) Generate
Input [7]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, _extract_quantityDenominator#71133, quantityNumerator#70019]
Arguments: explode(_extract_quantityDenominator#71133), [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, quantityNumerator#70019], true, [quantityDenominator#70020]
(553) Project
Output [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, (cast(quantityNumerator#70019 as double) / cast(quantityDenominator#70020 as double)) AS outboundUnit#70021]
Input [7]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, quantityNumerator#70019, quantityDenominator#70020]
(554) Project
Output [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, CASE WHEN (isnull(outboundUnit#70021) OR (outboundUnit#70021 <= 0.0)) THEN 1.0 ELSE outboundUnit#70021 END AS outboundUnit#70022]
Input [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70021]
(555) Sort
Input [6]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022]
Arguments: [product#69978 ASC NULLS FIRST, plant#69979 ASC NULLS FIRST], false, 0
(556) Scan parquet
Output [6]: [internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, stockQuantities#70028, systemId#70023]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(557) Filter
Input [6]: [internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, stockQuantities#70028, systemId#70023]
Condition : (isnotnull(plant#70026.internalRefUUID) AND isnotnull(product#70025.internalRefUUID))
(558) Project
Output [6]: [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, stockQuantities#70028]
Input [6]: [internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, stockQuantities#70028, systemId#70023]
(559) Exchange
Input [6]: [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, stockQuantities#70028]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31367]
(560) BroadcastExchange
Input [6]: [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, stockQuantities#70028]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=32381]
(561) Scan parquet
Output [2]: [marketUnit#70029, distributionCenters#70035]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(562) Filter
Input [2]: [marketUnit#70029, distributionCenters#70035]
Condition : (((marketUnit#70029 <=> Daily_grocery) AND (size(distributionCenters#70035, true) > 0)) AND isnotnull(distributionCenters#70035))
(563) Project
Output [1]: [distributionCenters#70035]
Input [2]: [marketUnit#70029, distributionCenters#70035]
(564) Exchange
Input [1]: [distributionCenters#70035]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31378]
(565) Generate
Input [1]: [distributionCenters#70035]
Arguments: explode(distributionCenters#70035), false, [distributionCenters#70051]
(566) Filter
Input [1]: [distributionCenters#70051]
Condition : (((size(distributionCenters#70051.storageLocations, true) > 0) AND isnotnull(distributionCenters#70051.storageLocations)) AND isnotnull(distributionCenters#70051.internalRefUUID))
(567) Project
Output [2]: [distributionCenters#70051.internalRefUUID AS _extract_internalRefUUID#71021, distributionCenters#70051.storageLocations AS _extract_storageLocations#71022]
Input [1]: [distributionCenters#70051]
(568) Generate
Input [2]: [_extract_internalRefUUID#71021, _extract_storageLocations#71022]
Arguments: explode(_extract_storageLocations#71022), [_extract_internalRefUUID#71021], false, [storageLocations#70069]
(569) Project
Output [3]: [_extract_internalRefUUID#71021 AS plantUuid#70071, storageLocations#70069.id AS storageLocId#70072, storageLocations#70069.internalRefUUID AS storageLocUuid#70073]
Input [2]: [_extract_internalRefUUID#71021, storageLocations#70069]
(570) BroadcastExchange
Input [3]: [plantUuid#70071, storageLocId#70072, storageLocUuid#70073]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32371]
(571) Scan parquet
Output [2]: [product#70074, plant#70075]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(572) Filter
Input [2]: [product#70074, plant#70075]
Condition : isnotnull(plant#70075)
(573) Exchange
Input [2]: [product#70074, plant#70075]
Arguments: hashpartitioning(product#70074, plant#70075, 25), REPARTITION_BY_NUM, [plan_id=31809]
(574) Scan parquet
Output [4]: [plant#70080, product#70081, productionAspect#70083, systemId#70078]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(575) Project
Output [4]: [plant#70080.internalRefUUID AS _extract_internalRefUUID#71405, product#70081.internalRefUUID AS _extract_internalRefUUID#71406, productionAspect#70083.productMovementPlants.unitOfIssue.code AS _extract_code#71403, productionAspect#70083.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71404]
Input [4]: [plant#70080, product#70081, productionAspect#70083, systemId#70078]
(576) Exchange
Input [4]: [_extract_internalRefUUID#71405, _extract_internalRefUUID#71406, _extract_code#71403, _extract_internalRefUUID#71404]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31407]
(577) Project
Output [4]: [_extract_code#71403, _extract_internalRefUUID#71404, _extract_internalRefUUID#71406, _extract_internalRefUUID#71405]
Input [4]: [_extract_internalRefUUID#71405, _extract_internalRefUUID#71406, _extract_code#71403, _extract_internalRefUUID#71404]
(578) BroadcastExchange
Input [4]: [_extract_code#71403, _extract_internalRefUUID#71404, _extract_internalRefUUID#71406, _extract_internalRefUUID#71405]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32336]
(579) BroadcastHashJoin
Left keys [4]: [coalesce(product#70074, ), isnull(product#70074), coalesce(plant#70075, ), isnull(plant#70075)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71406, ), isnull(_extract_internalRefUUID#71406), coalesce(_extract_internalRefUUID#71405, ), isnull(_extract_internalRefUUID#71405)]
Join type: LeftOuter
Join condition: None
(580) Project
Output [4]: [product#70074, plant#70075, _extract_code#71403, _extract_internalRefUUID#71404]
Input [6]: [product#70074, plant#70075, _extract_code#71403, _extract_internalRefUUID#71404, _extract_internalRefUUID#71406, _extract_internalRefUUID#71405]
(581) Scan parquet
Output [2]: [internalUUID#70088, systemId#70087]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(582) Project
Output [1]: [internalUUID#70088]
Input [2]: [internalUUID#70088, systemId#70087]
(583) Exchange
Input [1]: [internalUUID#70088]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31416]
(584) BroadcastExchange
Input [1]: [internalUUID#70088]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32344]
(585) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70075, ), isnull(plant#70075)]
Right keys [2]: [coalesce(internalUUID#70088, ), isnull(internalUUID#70088)]
Join type: LeftOuter
Join condition: None
(586) Project
Output [4]: [product#70074, plant#70075, _extract_code#71403, _extract_internalRefUUID#71404]
Input [5]: [product#70074, plant#70075, _extract_code#71403, _extract_internalRefUUID#71404, internalUUID#70088]
(587) Scan parquet
Output [4]: [internalUUID#70102, baseUnitOfMeasure#70106, unitOfMeasureConversions#70108, systemId#70101]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(588) Filter
Input [4]: [internalUUID#70102, baseUnitOfMeasure#70106, unitOfMeasureConversions#70108, systemId#70101]
Condition : isnotnull(internalUUID#70102)
(589) Project
Output [3]: [internalUUID#70102, baseUnitOfMeasure#70106, unitOfMeasureConversions#70108]
Input [4]: [internalUUID#70102, baseUnitOfMeasure#70106, unitOfMeasureConversions#70108, systemId#70101]
(590) Exchange
Input [3]: [internalUUID#70102, baseUnitOfMeasure#70106, unitOfMeasureConversions#70108]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31428]
(591) BroadcastExchange
Input [3]: [internalUUID#70102, baseUnitOfMeasure#70106, unitOfMeasureConversions#70108]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32349]
(592) BroadcastHashJoin
Left keys [1]: [product#70074]
Right keys [1]: [internalUUID#70102]
Join type: LeftOuter
Join condition: None
(593) Project
Output [2]: [plant#70075, filter(unitOfMeasureConversions#70108, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71403) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71404)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#70106.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70106.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#70113]
Input [7]: [product#70074, plant#70075, _extract_code#71403, _extract_internalRefUUID#71404, internalUUID#70102, baseUnitOfMeasure#70106, unitOfMeasureConversions#70108]
(594) Generate
Input [2]: [plant#70075, filteredUnitOfMeasureConversions#70113]
Arguments: explode(filteredUnitOfMeasureConversions#70113.quantityNumerator), [plant#70075, filteredUnitOfMeasureConversions#70113], true, [quantityNumerator#70114]
(595) Project
Output [2]: [plant#70075, filteredUnitOfMeasureConversions#70113.quantityDenominator AS _extract_quantityDenominator#71024]
Input [3]: [plant#70075, filteredUnitOfMeasureConversions#70113, quantityNumerator#70114]
(596) Generate
Input [2]: [plant#70075, _extract_quantityDenominator#71024]
Arguments: explode(_extract_quantityDenominator#71024), [plant#70075], true, [quantityDenominator#70115]
(597) Project
Output [1]: [plant#70075]
Input [2]: [plant#70075, quantityDenominator#70115]
(598) HashAggregate
Input [1]: [plant#70075]
Keys [1]: [plant#70075]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70075]
(599) Exchange
Input [1]: [plant#70075]
Arguments: hashpartitioning(plant#70075, 25), ENSURE_REQUIREMENTS, [plan_id=32365]
(600) HashAggregate
Input [1]: [plant#70075]
Keys [1]: [plant#70075]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70075]
(601) BroadcastHashJoin
Left keys [1]: [plantUuid#70071]
Right keys [1]: [plant#70075]
Join type: Inner
Join condition: None
(602) Project
Output [3]: [plantUuid#70071, storageLocId#70072, storageLocUuid#70073]
Input [4]: [plantUuid#70071, storageLocId#70072, storageLocUuid#70073, plant#70075]
(603) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70026.internalRefUUID, ), isnull(plant#70026.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#70071, ), isnull(plantUuid#70071)]
Join type: Inner
Join condition: None
(604) Generate
Input [9]: [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, stockQuantities#70028, plantUuid#70071, storageLocId#70072, storageLocUuid#70073]
Arguments: explode(filter(stockQuantities#70028, lambdafunction(((lambda x#69269.storageLocation.internalRefUuid = storageLocUuid#70073) AND (lambda x#69269.storageLocation.Id = storageLocId#70072)), lambda x#69269, false))), [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, plantUuid#70071], false, [stockQuantity#70118]
(605) ObjectHashAggregate
Input [7]: [systemId#70023, internalUUID#70024, product#70025, plant#70026, calculationDateTime#70027, plantUuid#70071, stockQuantity#70118]
Keys [5]: [plantUuid#70071, internalUUID#70024, plant#70026, product#70025, systemId#70023]
Functions [2]: [partial_first(calculationDateTime#70027, false), partial_collect_list(stockQuantity#70118, 0, 0)]
Aggregate Attributes [3]: [first#73962, valueSet#73963, buf#73964]
Results [8]: [plantUuid#70071, internalUUID#70024, plant#70026, product#70025, systemId#70023, first#73965, valueSet#73966, buf#73967]
(606) ObjectHashAggregate
Input [8]: [plantUuid#70071, internalUUID#70024, plant#70026, product#70025, systemId#70023, first#73965, valueSet#73966, buf#73967]
Keys [5]: [plantUuid#70071, internalUUID#70024, plant#70026, product#70025, systemId#70023]
Functions [2]: [first(calculationDateTime#70027, false), collect_list(stockQuantity#70118, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#70027)()#69286, collect_list(stockQuantity#70118, 0, 0)#69287]
Results [4]: [product#70025, plant#70026, first(calculationDateTime#70027)()#69286 AS calculationDateTime#70119, collect_list(stockQuantity#70118, 0, 0)#69287 AS stockQuantities#70120]
(607) Filter
Input [4]: [product#70025, plant#70026, calculationDateTime#70119, stockQuantities#70120]
Condition : ((size(stockQuantities#70120, true) > 0) AND isnotnull(calculationDateTime#70119))
(608) Generate
Input [4]: [product#70025, plant#70026, calculationDateTime#70119, stockQuantities#70120]
Arguments: explode(stockQuantities#70120), [product#70025, plant#70026, calculationDateTime#70119], false, [stockQuantity#70121]
(609) Filter
Input [4]: [product#70025, plant#70026, calculationDateTime#70119, stockQuantity#70121]
Condition : (((isnotnull(stockQuantity#70121.specialStockIndicator.code) AND isnotnull(stockQuantity#70121.stockType.code)) AND (stockQuantity#70121.specialStockIndicator.code = )) AND (stockQuantity#70121.stockType.code = 01))
(610) Project
Output [5]: [plant#70026.internalRefUUID AS plantUuid#70123, product#70025.internalRefUUID AS productUuid#70125, calculationDateTime#70119, stockQuantity#70121.storagelocation.id AS storageLocId#70126, stockQuantity#70121.quantity.measure AS quantity#70128]
Input [4]: [product#70025, plant#70026, calculationDateTime#70119, stockQuantity#70121]
(611) Exchange
Input [5]: [plantUuid#70123, productUuid#70125, calculationDateTime#70119, storageLocId#70126, quantity#70128]
Arguments: hashpartitioning(productUuid#70125, plantUuid#70123, 25), ENSURE_REQUIREMENTS, [plan_id=32442]
(612) Sort
Input [5]: [plantUuid#70123, productUuid#70125, calculationDateTime#70119, storageLocId#70126, quantity#70128]
Arguments: [productUuid#70125 ASC NULLS FIRST, plantUuid#70123 ASC NULLS FIRST], false, 0
(613) Scan parquet
Output [3]: [product#70131, plant#70132, orderDateTime#70133]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(614) Filter
Input [3]: [product#70131, plant#70132, orderDateTime#70133]
Condition : ((isnotnull(orderDateTime#70133) AND isnotnull(plant#70132)) AND isnotnull(product#70131))
(615) Exchange
Input [3]: [product#70131, plant#70132, orderDateTime#70133]
Arguments: hashpartitioning(product#70131, plant#70132, 25), REPARTITION_BY_NUM, [plan_id=31855]
(616) Scan parquet
Output [4]: [plant#70137, product#70138, productionAspect#70140, systemId#70135]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(617) Project
Output [4]: [plant#70137.internalRefUUID AS _extract_internalRefUUID#71407, product#70138.internalRefUUID AS _extract_internalRefUUID#71410, productionAspect#70140.productMovementPlants.unitOfIssue.code AS _extract_code#71408, productionAspect#70140.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71409]
Input [4]: [plant#70137, product#70138, productionAspect#70140, systemId#70135]
(618) Exchange
Input [4]: [_extract_internalRefUUID#71407, _extract_internalRefUUID#71410, _extract_code#71408, _extract_internalRefUUID#71409]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31456]
(619) Project
Output [4]: [_extract_code#71408, _extract_internalRefUUID#71409, _extract_internalRefUUID#71410, _extract_internalRefUUID#71407]
Input [4]: [_extract_internalRefUUID#71407, _extract_internalRefUUID#71410, _extract_code#71408, _extract_internalRefUUID#71409]
(620) BroadcastExchange
Input [4]: [_extract_code#71408, _extract_internalRefUUID#71409, _extract_internalRefUUID#71410, _extract_internalRefUUID#71407]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32407]
(621) BroadcastHashJoin
Left keys [4]: [coalesce(product#70131, ), isnull(product#70131), coalesce(plant#70132, ), isnull(plant#70132)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71410, ), isnull(_extract_internalRefUUID#71410), coalesce(_extract_internalRefUUID#71407, ), isnull(_extract_internalRefUUID#71407)]
Join type: LeftOuter
Join condition: None
(622) Project
Output [5]: [product#70131, plant#70132, orderDateTime#70133, _extract_code#71408, _extract_internalRefUUID#71409]
Input [7]: [product#70131, plant#70132, orderDateTime#70133, _extract_code#71408, _extract_internalRefUUID#71409, _extract_internalRefUUID#71410, _extract_internalRefUUID#71407]
(623) Scan parquet
Output [2]: [internalUUID#70145, systemId#70144]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(624) Project
Output [1]: [internalUUID#70145]
Input [2]: [internalUUID#70145, systemId#70144]
(625) Exchange
Input [1]: [internalUUID#70145]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31462]
(626) BroadcastExchange
Input [1]: [internalUUID#70145]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32412]
(627) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70132, ), isnull(plant#70132)]
Right keys [2]: [coalesce(internalUUID#70145, ), isnull(internalUUID#70145)]
Join type: LeftOuter
Join condition: None
(628) Project
Output [5]: [product#70131, plant#70132, orderDateTime#70133, _extract_code#71408, _extract_internalRefUUID#71409]
Input [6]: [product#70131, plant#70132, orderDateTime#70133, _extract_code#71408, _extract_internalRefUUID#71409, internalUUID#70145]
(629) Scan parquet
Output [4]: [internalUUID#70159, baseUnitOfMeasure#70163, unitOfMeasureConversions#70165, systemId#70158]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(630) Filter
Input [4]: [internalUUID#70159, baseUnitOfMeasure#70163, unitOfMeasureConversions#70165, systemId#70158]
Condition : isnotnull(internalUUID#70159)
(631) Project
Output [3]: [internalUUID#70159, baseUnitOfMeasure#70163, unitOfMeasureConversions#70165]
Input [4]: [internalUUID#70159, baseUnitOfMeasure#70163, unitOfMeasureConversions#70165, systemId#70158]
(632) Exchange
Input [3]: [internalUUID#70159, baseUnitOfMeasure#70163, unitOfMeasureConversions#70165]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31468]
(633) BroadcastExchange
Input [3]: [internalUUID#70159, baseUnitOfMeasure#70163, unitOfMeasureConversions#70165]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32418]
(634) BroadcastHashJoin
Left keys [1]: [product#70131]
Right keys [1]: [internalUUID#70159]
Join type: LeftOuter
Join condition: None
(635) Project
Output [4]: [product#70131, plant#70132, orderDateTime#70133, filter(unitOfMeasureConversions#70165, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71408) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71409)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#70163.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70163.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#70171]
Input [8]: [product#70131, plant#70132, orderDateTime#70133, _extract_code#71408, _extract_internalRefUUID#71409, internalUUID#70159, baseUnitOfMeasure#70163, unitOfMeasureConversions#70165]
(636) Generate
Input [4]: [product#70131, plant#70132, orderDateTime#70133, filteredUnitOfMeasureConversions#70171]
Arguments: explode(filteredUnitOfMeasureConversions#70171.quantityNumerator), [product#70131, plant#70132, orderDateTime#70133, filteredUnitOfMeasureConversions#70171], true, [quantityNumerator#70172]
(637) Project
Output [4]: [product#70131, plant#70132, orderDateTime#70133, filteredUnitOfMeasureConversions#70171.quantityDenominator AS _extract_quantityDenominator#71031]
Input [5]: [product#70131, plant#70132, orderDateTime#70133, filteredUnitOfMeasureConversions#70171, quantityNumerator#70172]
(638) Generate
Input [4]: [product#70131, plant#70132, orderDateTime#70133, _extract_quantityDenominator#71031]
Arguments: explode(_extract_quantityDenominator#71031), [product#70131, plant#70132, orderDateTime#70133], true, [quantityDenominator#70173]
(639) Project
Output [3]: [product#70131, plant#70132, orderDateTime#70133]
Input [4]: [product#70131, plant#70132, orderDateTime#70133, quantityDenominator#70173]
(640) Sort
Input [3]: [product#70131, plant#70132, orderDateTime#70133]
Arguments: [product#70131 ASC NULLS FIRST, plant#70132 ASC NULLS FIRST], false, 0
(641) SortMergeJoin
Left keys [2]: [productUuid#70125, plantUuid#70123]
Right keys [2]: [product#70131, plant#70132]
Join type: Inner
Join condition: (calculationDateTime#70119 <= orderDateTime#70133)
(642) Project
Output [5]: [plantUuid#70123 AS plant#70176, productUuid#70125 AS product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128]
Input [8]: [plantUuid#70123, productUuid#70125, calculationDateTime#70119, storageLocId#70126, quantity#70128, product#70131, plant#70132, orderDateTime#70133]
(643) Exchange
Input [5]: [plant#70176, product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128]
Arguments: hashpartitioning(coalesce(plant#70176, ), isnull(plant#70176), coalesce(product#70177, ), isnull(product#70177), coalesce(storageLocId#70126, ), isnull(storageLocId#70126), coalesce(calculationDateTime#70119, 1970-01-01 00:00:00), isnull(calculationDateTime#70119), 25), ENSURE_REQUIREMENTS, [plan_id=32583]
(644) Sort
Input [5]: [plant#70176, product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128]
Arguments: [coalesce(plant#70176, ) ASC NULLS FIRST, isnull(plant#70176) ASC NULLS FIRST, coalesce(product#70177, ) ASC NULLS FIRST, isnull(product#70177) ASC NULLS FIRST, coalesce(storageLocId#70126, ) ASC NULLS FIRST, isnull(storageLocId#70126) ASC NULLS FIRST, coalesce(calculationDateTime#70119, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(calculationDateTime#70119) ASC NULLS FIRST], false, 0
(645) Scan parquet
Output [6]: [internalUUID#70179, product#70180, plant#70181, calculationDateTime#70182, stockQuantities#70183, systemId#70178]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-inventory-inventorysnapshot-v2]
PushedFilters: [IsNotNull(plant.internalRefUUID), IsNotNull(product.internalRefUUID)]
ReadSchema: struct<internalUUID:string,product:struct<internalRefUUID:string,id:string>,plant:struct<internalRefUUID:string,id:string>,calculationDateTime:timestamp,stockQuantities:array<struct<stockType:struct<internalRefUUID:string,code:string>,specialStockIndicator:struct<internalRefUUID:string,code:string>,storageLocation:struct<internalRefUUID:string,id:string,plant:struct<internalRefUUID:string,id:string>>,quantity:struct<measure:double,unitCode:struct<internalRefUUID:string,code:string>>>>>
(646) Filter
Input [6]: [internalUUID#70179, product#70180, plant#70181, calculationDateTime#70182, stockQuantities#70183, systemId#70178]
Condition : (isnotnull(plant#70181.internalRefUUID) AND isnotnull(product#70180.internalRefUUID))
(647) Project
Output [6]: [systemId#70178, internalUUID#70179, product#70180, plant#70181, calculationDateTime#70182, stockQuantities#70183]
Input [6]: [internalUUID#70179, product#70180, plant#70181, calculationDateTime#70182, stockQuantities#70183, systemId#70178]
(648) Exchange
Input [6]: [systemId#70178, internalUUID#70179, product#70180, plant#70181, calculationDateTime#70182, stockQuantities#70183]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31480]
(649) BroadcastExchange
Input [6]: [systemId#70178, internalUUID#70179, product#70180, plant#70181, calculationDateTime#70182, stockQuantities#70183]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID, ), isnull(input[3, struct<internalRefUUID:string,id:string>, true].internalRefUUID)),false), [plan_id=32503]
(650) Scan parquet
Output [2]: [marketUnit#70184, distributionCenters#70190]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(651) Filter
Input [2]: [marketUnit#70184, distributionCenters#70190]
Condition : (((marketUnit#70184 <=> Daily_grocery) AND (size(distributionCenters#70190, true) > 0)) AND isnotnull(distributionCenters#70190))
(652) Project
Output [1]: [distributionCenters#70190]
Input [2]: [marketUnit#70184, distributionCenters#70190]
(653) Exchange
Input [1]: [distributionCenters#70190]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31484]
(654) Generate
Input [1]: [distributionCenters#70190]
Arguments: explode(distributionCenters#70190), false, [distributionCenters#70206]
(655) Filter
Input [1]: [distributionCenters#70206]
Condition : (((size(distributionCenters#70206.storageLocations, true) > 0) AND isnotnull(distributionCenters#70206.storageLocations)) AND isnotnull(distributionCenters#70206.internalRefUUID))
(656) Project
Output [2]: [distributionCenters#70206.internalRefUUID AS _extract_internalRefUUID#71041, distributionCenters#70206.storageLocations AS _extract_storageLocations#71042]
Input [1]: [distributionCenters#70206]
(657) Generate
Input [2]: [_extract_internalRefUUID#71041, _extract_storageLocations#71042]
Arguments: explode(_extract_storageLocations#71042), [_extract_internalRefUUID#71041], false, [storageLocations#70207]
(658) Project
Output [3]: [_extract_internalRefUUID#71041 AS plantUuid#70209, storageLocations#70207.id AS storageLocId#70210, storageLocations#70207.internalRefUUID AS storageLocUuid#70211]
Input [2]: [_extract_internalRefUUID#71041, storageLocations#70207]
(659) BroadcastExchange
Input [3]: [plantUuid#70209, storageLocId#70210, storageLocUuid#70211]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32494]
(660) Scan parquet
Output [2]: [product#70212, plant#70213]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(661) Filter
Input [2]: [product#70212, plant#70213]
Condition : isnotnull(plant#70213)
(662) Exchange
Input [2]: [product#70212, plant#70213]
Arguments: hashpartitioning(product#70212, plant#70213, 25), REPARTITION_BY_NUM, [plan_id=31881]
(663) Scan parquet
Output [4]: [plant#70218, product#70219, productionAspect#70221, systemId#70216]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(664) Project
Output [4]: [plant#70218.internalRefUUID AS _extract_internalRefUUID#71422, product#70219.internalRefUUID AS _extract_internalRefUUID#71419, productionAspect#70221.productMovementPlants.unitOfIssue.code AS _extract_code#71420, productionAspect#70221.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71421]
Input [4]: [plant#70218, product#70219, productionAspect#70221, systemId#70216]
(665) Exchange
Input [4]: [_extract_internalRefUUID#71422, _extract_internalRefUUID#71419, _extract_code#71420, _extract_internalRefUUID#71421]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31496]
(666) Project
Output [4]: [_extract_code#71420, _extract_internalRefUUID#71421, _extract_internalRefUUID#71419, _extract_internalRefUUID#71422]
Input [4]: [_extract_internalRefUUID#71422, _extract_internalRefUUID#71419, _extract_code#71420, _extract_internalRefUUID#71421]
(667) BroadcastExchange
Input [4]: [_extract_code#71420, _extract_internalRefUUID#71421, _extract_internalRefUUID#71419, _extract_internalRefUUID#71422]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32454]
(668) BroadcastHashJoin
Left keys [4]: [coalesce(product#70212, ), isnull(product#70212), coalesce(plant#70213, ), isnull(plant#70213)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71419, ), isnull(_extract_internalRefUUID#71419), coalesce(_extract_internalRefUUID#71422, ), isnull(_extract_internalRefUUID#71422)]
Join type: LeftOuter
Join condition: None
(669) Project
Output [4]: [product#70212, plant#70213, _extract_code#71420, _extract_internalRefUUID#71421]
Input [6]: [product#70212, plant#70213, _extract_code#71420, _extract_internalRefUUID#71421, _extract_internalRefUUID#71419, _extract_internalRefUUID#71422]
(670) Scan parquet
Output [2]: [internalUUID#70226, systemId#70225]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(671) Project
Output [1]: [internalUUID#70226]
Input [2]: [internalUUID#70226, systemId#70225]
(672) Exchange
Input [1]: [internalUUID#70226]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31502]
(673) BroadcastExchange
Input [1]: [internalUUID#70226]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32461]
(674) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70213, ), isnull(plant#70213)]
Right keys [2]: [coalesce(internalUUID#70226, ), isnull(internalUUID#70226)]
Join type: LeftOuter
Join condition: None
(675) Project
Output [4]: [product#70212, plant#70213, _extract_code#71420, _extract_internalRefUUID#71421]
Input [5]: [product#70212, plant#70213, _extract_code#71420, _extract_internalRefUUID#71421, internalUUID#70226]
(676) Scan parquet
Output [4]: [internalUUID#70240, baseUnitOfMeasure#70244, unitOfMeasureConversions#70246, systemId#70239]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(677) Filter
Input [4]: [internalUUID#70240, baseUnitOfMeasure#70244, unitOfMeasureConversions#70246, systemId#70239]
Condition : isnotnull(internalUUID#70240)
(678) Project
Output [3]: [internalUUID#70240, baseUnitOfMeasure#70244, unitOfMeasureConversions#70246]
Input [4]: [internalUUID#70240, baseUnitOfMeasure#70244, unitOfMeasureConversions#70246, systemId#70239]
(679) Exchange
Input [3]: [internalUUID#70240, baseUnitOfMeasure#70244, unitOfMeasureConversions#70246]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31508]
(680) BroadcastExchange
Input [3]: [internalUUID#70240, baseUnitOfMeasure#70244, unitOfMeasureConversions#70246]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32469]
(681) BroadcastHashJoin
Left keys [1]: [product#70212]
Right keys [1]: [internalUUID#70240]
Join type: LeftOuter
Join condition: None
(682) Project
Output [2]: [plant#70213, filter(unitOfMeasureConversions#70246, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71420) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71421)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#70244.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70244.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#70251]
Input [7]: [product#70212, plant#70213, _extract_code#71420, _extract_internalRefUUID#71421, internalUUID#70240, baseUnitOfMeasure#70244, unitOfMeasureConversions#70246]
(683) Generate
Input [2]: [plant#70213, filteredUnitOfMeasureConversions#70251]
Arguments: explode(filteredUnitOfMeasureConversions#70251.quantityNumerator), [plant#70213, filteredUnitOfMeasureConversions#70251], true, [quantityNumerator#70252]
(684) Project
Output [2]: [plant#70213, filteredUnitOfMeasureConversions#70251.quantityDenominator AS _extract_quantityDenominator#71044]
Input [3]: [plant#70213, filteredUnitOfMeasureConversions#70251, quantityNumerator#70252]
(685) Generate
Input [2]: [plant#70213, _extract_quantityDenominator#71044]
Arguments: explode(_extract_quantityDenominator#71044), [plant#70213], true, [quantityDenominator#70253]
(686) Project
Output [1]: [plant#70213]
Input [2]: [plant#70213, quantityDenominator#70253]
(687) HashAggregate
Input [1]: [plant#70213]
Keys [1]: [plant#70213]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70213]
(688) Exchange
Input [1]: [plant#70213]
Arguments: hashpartitioning(plant#70213, 25), ENSURE_REQUIREMENTS, [plan_id=32488]
(689) HashAggregate
Input [1]: [plant#70213]
Keys [1]: [plant#70213]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70213]
(690) BroadcastHashJoin
Left keys [1]: [plantUuid#70209]
Right keys [1]: [plant#70213]
Join type: Inner
Join condition: None
(691) Project
Output [3]: [plantUuid#70209, storageLocId#70210, storageLocUuid#70211]
Input [4]: [plantUuid#70209, storageLocId#70210, storageLocUuid#70211, plant#70213]
(692) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70181.internalRefUUID, ), isnull(plant#70181.internalRefUUID)]
Right keys [2]: [coalesce(plantUuid#70209, ), isnull(plantUuid#70209)]
Join type: Inner
Join condition: None
(693) Generate
Input [9]: [systemId#70178, internalUUID#70179, product#70180, plant#70181, calculationDateTime#70182, stockQuantities#70183, plantUuid#70209, storageLocId#70210, storageLocUuid#70211]
Arguments: explode(filter(stockQuantities#70183, lambdafunction(((lambda x#69269.storageLocation.internalRefUuid = storageLocUuid#70211) AND (lambda x#69269.storageLocation.Id = storageLocId#70210)), lambda x#69269, false))), [systemId#70178, internalUUID#70179, product#70180, plant#70181, calculationDateTime#70182, plantUuid#70209], false, [stockQuantity#70256]
(694) ObjectHashAggregate
Input [7]: [systemId#70178, internalUUID#70179, product#70180, plant#70181, calculationDateTime#70182, plantUuid#70209, stockQuantity#70256]
Keys [5]: [plantUuid#70209, internalUUID#70179, plant#70181, product#70180, systemId#70178]
Functions [2]: [partial_first(calculationDateTime#70182, false), partial_collect_list(stockQuantity#70256, 0, 0)]
Aggregate Attributes [3]: [first#73976, valueSet#73977, buf#73978]
Results [8]: [plantUuid#70209, internalUUID#70179, plant#70181, product#70180, systemId#70178, first#73979, valueSet#73980, buf#73981]
(695) ObjectHashAggregate
Input [8]: [plantUuid#70209, internalUUID#70179, plant#70181, product#70180, systemId#70178, first#73979, valueSet#73980, buf#73981]
Keys [5]: [plantUuid#70209, internalUUID#70179, plant#70181, product#70180, systemId#70178]
Functions [2]: [first(calculationDateTime#70182, false), collect_list(stockQuantity#70256, 0, 0)]
Aggregate Attributes [2]: [first(calculationDateTime#70182)()#69286, collect_list(stockQuantity#70256, 0, 0)#69287]
Results [4]: [product#70180, plant#70181, first(calculationDateTime#70182)()#69286 AS calculationDateTime#70257, collect_list(stockQuantity#70256, 0, 0)#69287 AS stockQuantities#70258]
(696) Filter
Input [4]: [product#70180, plant#70181, calculationDateTime#70257, stockQuantities#70258]
Condition : ((size(stockQuantities#70258, true) > 0) AND isnotnull(calculationDateTime#70257))
(697) Generate
Input [4]: [product#70180, plant#70181, calculationDateTime#70257, stockQuantities#70258]
Arguments: explode(stockQuantities#70258), [product#70180, plant#70181, calculationDateTime#70257], false, [stockQuantity#70259]
(698) Filter
Input [4]: [product#70180, plant#70181, calculationDateTime#70257, stockQuantity#70259]
Condition : (((isnotnull(stockQuantity#70259.specialStockIndicator.code) AND isnotnull(stockQuantity#70259.stockType.code)) AND (stockQuantity#70259.specialStockIndicator.code = )) AND (stockQuantity#70259.stockType.code = 01))
(699) Project
Output [4]: [plant#70181.internalRefUUID AS plantUuid#70261, product#70180.internalRefUUID AS productUuid#70263, calculationDateTime#70257, stockQuantity#70259.storagelocation.id AS storageLocId#70264]
Input [4]: [product#70180, plant#70181, calculationDateTime#70257, stockQuantity#70259]
(700) Exchange
Input [4]: [plantUuid#70261, productUuid#70263, calculationDateTime#70257, storageLocId#70264]
Arguments: hashpartitioning(productUuid#70263, plantUuid#70261, 25), ENSURE_REQUIREMENTS, [plan_id=32561]
(701) Sort
Input [4]: [plantUuid#70261, productUuid#70263, calculationDateTime#70257, storageLocId#70264]
Arguments: [productUuid#70263 ASC NULLS FIRST, plantUuid#70261 ASC NULLS FIRST], false, 0
(702) Scan parquet
Output [3]: [product#70269, plant#70270, orderDateTime#70271]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
PushedFilters: [IsNotNull(orderDateTime), IsNotNull(plant), IsNotNull(product)]
ReadSchema: struct<product:string,plant:string,orderDateTime:timestamp>
(703) Filter
Input [3]: [product#70269, plant#70270, orderDateTime#70271]
Condition : ((isnotnull(orderDateTime#70271) AND isnotnull(plant#70270)) AND isnotnull(product#70269))
(704) Exchange
Input [3]: [product#70269, plant#70270, orderDateTime#70271]
Arguments: hashpartitioning(product#70269, plant#70270, 25), REPARTITION_BY_NUM, [plan_id=31927]
(705) Scan parquet
Output [4]: [plant#70275, product#70276, productionAspect#70278, systemId#70273]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(706) Project
Output [4]: [plant#70275.internalRefUUID AS _extract_internalRefUUID#71425, product#70276.internalRefUUID AS _extract_internalRefUUID#71426, productionAspect#70278.productMovementPlants.unitOfIssue.code AS _extract_code#71423, productionAspect#70278.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71424]
Input [4]: [plant#70275, product#70276, productionAspect#70278, systemId#70273]
(707) Exchange
Input [4]: [_extract_internalRefUUID#71425, _extract_internalRefUUID#71426, _extract_code#71423, _extract_internalRefUUID#71424]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31533]
(708) Project
Output [4]: [_extract_code#71423, _extract_internalRefUUID#71424, _extract_internalRefUUID#71426, _extract_internalRefUUID#71425]
Input [4]: [_extract_internalRefUUID#71425, _extract_internalRefUUID#71426, _extract_code#71423, _extract_internalRefUUID#71424]
(709) BroadcastExchange
Input [4]: [_extract_code#71423, _extract_internalRefUUID#71424, _extract_internalRefUUID#71426, _extract_internalRefUUID#71425]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32521]
(710) BroadcastHashJoin
Left keys [4]: [coalesce(product#70269, ), isnull(product#70269), coalesce(plant#70270, ), isnull(plant#70270)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71426, ), isnull(_extract_internalRefUUID#71426), coalesce(_extract_internalRefUUID#71425, ), isnull(_extract_internalRefUUID#71425)]
Join type: LeftOuter
Join condition: None
(711) Project
Output [5]: [product#70269, plant#70270, orderDateTime#70271, _extract_code#71423, _extract_internalRefUUID#71424]
Input [7]: [product#70269, plant#70270, orderDateTime#70271, _extract_code#71423, _extract_internalRefUUID#71424, _extract_internalRefUUID#71426, _extract_internalRefUUID#71425]
(712) Scan parquet
Output [2]: [internalUUID#70283, systemId#70282]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(713) Project
Output [1]: [internalUUID#70283]
Input [2]: [internalUUID#70283, systemId#70282]
(714) Exchange
Input [1]: [internalUUID#70283]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31539]
(715) BroadcastExchange
Input [1]: [internalUUID#70283]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32529]
(716) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70270, ), isnull(plant#70270)]
Right keys [2]: [coalesce(internalUUID#70283, ), isnull(internalUUID#70283)]
Join type: LeftOuter
Join condition: None
(717) Project
Output [5]: [product#70269, plant#70270, orderDateTime#70271, _extract_code#71423, _extract_internalRefUUID#71424]
Input [6]: [product#70269, plant#70270, orderDateTime#70271, _extract_code#71423, _extract_internalRefUUID#71424, internalUUID#70283]
(718) Scan parquet
Output [4]: [internalUUID#70297, baseUnitOfMeasure#70301, unitOfMeasureConversions#70303, systemId#70296]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(719) Filter
Input [4]: [internalUUID#70297, baseUnitOfMeasure#70301, unitOfMeasureConversions#70303, systemId#70296]
Condition : isnotnull(internalUUID#70297)
(720) Project
Output [3]: [internalUUID#70297, baseUnitOfMeasure#70301, unitOfMeasureConversions#70303]
Input [4]: [internalUUID#70297, baseUnitOfMeasure#70301, unitOfMeasureConversions#70303, systemId#70296]
(721) Exchange
Input [3]: [internalUUID#70297, baseUnitOfMeasure#70301, unitOfMeasureConversions#70303]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31545]
(722) BroadcastExchange
Input [3]: [internalUUID#70297, baseUnitOfMeasure#70301, unitOfMeasureConversions#70303]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32537]
(723) BroadcastHashJoin
Left keys [1]: [product#70269]
Right keys [1]: [internalUUID#70297]
Join type: LeftOuter
Join condition: None
(724) Project
Output [4]: [product#70269, plant#70270, orderDateTime#70271, filter(unitOfMeasureConversions#70303, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71423) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71424)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#70301.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70301.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#70308]
Input [8]: [product#70269, plant#70270, orderDateTime#70271, _extract_code#71423, _extract_internalRefUUID#71424, internalUUID#70297, baseUnitOfMeasure#70301, unitOfMeasureConversions#70303]
(725) Generate
Input [4]: [product#70269, plant#70270, orderDateTime#70271, filteredUnitOfMeasureConversions#70308]
Arguments: explode(filteredUnitOfMeasureConversions#70308.quantityNumerator), [product#70269, plant#70270, orderDateTime#70271, filteredUnitOfMeasureConversions#70308], true, [quantityNumerator#70309]
(726) Project
Output [4]: [product#70269, plant#70270, orderDateTime#70271, filteredUnitOfMeasureConversions#70308.quantityDenominator AS _extract_quantityDenominator#71052]
Input [5]: [product#70269, plant#70270, orderDateTime#70271, filteredUnitOfMeasureConversions#70308, quantityNumerator#70309]
(727) Generate
Input [4]: [product#70269, plant#70270, orderDateTime#70271, _extract_quantityDenominator#71052]
Arguments: explode(_extract_quantityDenominator#71052), [product#70269, plant#70270, orderDateTime#70271], true, [quantityDenominator#70310]
(728) Project
Output [3]: [product#70269, plant#70270, orderDateTime#70271]
Input [4]: [product#70269, plant#70270, orderDateTime#70271, quantityDenominator#70310]
(729) Sort
Input [3]: [product#70269, plant#70270, orderDateTime#70271]
Arguments: [product#70269 ASC NULLS FIRST, plant#70270 ASC NULLS FIRST], false, 0
(730) SortMergeJoin
Left keys [2]: [productUuid#70263, plantUuid#70261]
Right keys [2]: [product#70269, plant#70270]
Join type: Inner
Join condition: (calculationDateTime#70257 <= orderDateTime#70271)
(731) Project
Output [4]: [plantUuid#70261 AS plant#70323, productUuid#70263 AS product#70324, storageLocId#70264, calculationDateTime#70257]
Input [7]: [plantUuid#70261, productUuid#70263, calculationDateTime#70257, storageLocId#70264, product#70269, plant#70270, orderDateTime#70271]
(732) HashAggregate
Input [4]: [plant#70323, product#70324, storageLocId#70264, calculationDateTime#70257]
Keys [3]: [plant#70323, product#70324, storageLocId#70264]
Functions [1]: [partial_max(calculationDateTime#70257)]
Aggregate Attributes [1]: [max#73974]
Results [4]: [plant#70323, product#70324, storageLocId#70264, max#73975]
(733) HashAggregate
Input [4]: [plant#70323, product#70324, storageLocId#70264, max#73975]
Keys [3]: [plant#70323, product#70324, storageLocId#70264]
Functions [1]: [max(calculationDateTime#70257)]
Aggregate Attributes [1]: [max(calculationDateTime#70257)#69447]
Results [4]: [plant#70323, product#70324, storageLocId#70264, max(calculationDateTime#70257)#69447 AS max_calc_datetime#70325]
(734) Exchange
Input [4]: [plant#70323, product#70324, storageLocId#70264, max_calc_datetime#70325]
Arguments: hashpartitioning(coalesce(plant#70323, ), isnull(plant#70323), coalesce(product#70324, ), isnull(product#70324), coalesce(storageLocId#70264, ), isnull(storageLocId#70264), coalesce(max_calc_datetime#70325, 1970-01-01 00:00:00), isnull(max_calc_datetime#70325), 25), ENSURE_REQUIREMENTS, [plan_id=32584]
(735) Sort
Input [4]: [plant#70323, product#70324, storageLocId#70264, max_calc_datetime#70325]
Arguments: [coalesce(plant#70323, ) ASC NULLS FIRST, isnull(plant#70323) ASC NULLS FIRST, coalesce(product#70324, ) ASC NULLS FIRST, isnull(product#70324) ASC NULLS FIRST, coalesce(storageLocId#70264, ) ASC NULLS FIRST, isnull(storageLocId#70264) ASC NULLS FIRST, coalesce(max_calc_datetime#70325, 1970-01-01 00:00:00) ASC NULLS FIRST, isnull(max_calc_datetime#70325) ASC NULLS FIRST], false, 0
(736) SortMergeJoin
Left keys [8]: [coalesce(plant#70176, ), isnull(plant#70176), coalesce(product#70177, ), isnull(product#70177), coalesce(storageLocId#70126, ), isnull(storageLocId#70126), coalesce(calculationDateTime#70119, 1970-01-01 00:00:00), isnull(calculationDateTime#70119)]
Right keys [8]: [coalesce(plant#70323, ), isnull(plant#70323), coalesce(product#70324, ), isnull(product#70324), coalesce(storageLocId#70264, ), isnull(storageLocId#70264), coalesce(max_calc_datetime#70325, 1970-01-01 00:00:00), isnull(max_calc_datetime#70325)]
Join type: Inner
Join condition: None
(737) Project
Output [4]: [plant#70176, product#70177, calculationDateTime#70119, quantity#70128]
Input [9]: [plant#70176, product#70177, storageLocId#70126, calculationDateTime#70119, quantity#70128, plant#70323, product#70324, storageLocId#70264, max_calc_datetime#70325]
(738) HashAggregate
Input [4]: [plant#70176, product#70177, calculationDateTime#70119, quantity#70128]
Keys [2]: [product#70177, plant#70176]
Functions [2]: [partial_min(calculationDateTime#70119), partial_sum(quantity#70128)]
Aggregate Attributes [2]: [min#73956, sum#73957]
Results [4]: [product#70177, plant#70176, min#73958, sum#73959]
(739) Exchange
Input [4]: [product#70177, plant#70176, min#73958, sum#73959]
Arguments: hashpartitioning(product#70177, plant#70176, 25), ENSURE_REQUIREMENTS, [plan_id=32596]
(740) HashAggregate
Input [4]: [product#70177, plant#70176, min#73958, sum#73959]
Keys [2]: [product#70177, plant#70176]
Functions [2]: [min(calculationDateTime#70119), sum(quantity#70128)]
Aggregate Attributes [2]: [min(calculationDateTime#70119)#69766, sum(quantity#70128)#69765]
Results [4]: [product#70177, plant#70176, min(calculationDateTime#70119)#69766 AS minCalculationDateTime#70327, round(sum(quantity#70128)#69765, 0) AS roundedQuantitySum#70328]
(741) Sort
Input [4]: [product#70177, plant#70176, minCalculationDateTime#70327, roundedQuantitySum#70328]
Arguments: [product#70177 ASC NULLS FIRST, plant#70176 ASC NULLS FIRST], false, 0
(742) SortMergeJoin
Left keys [2]: [product#69978, plant#69979]
Right keys [2]: [product#70177, plant#70176]
Join type: LeftOuter
Join condition: None
(743) Project
Output [8]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, CASE WHEN isnotnull(minCalculationDateTime#70327) THEN minCalculationDateTime#70327 ELSE orderDateTime#69980 END AS stockDatetime#70329, CASE WHEN isnotnull(roundedQuantitySum#70328) THEN array(roundedQuantitySum#70328) ELSE [0.0] END AS stockBatchQuantities#70330]
Input [10]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, product#70177, plant#70176, minCalculationDateTime#70327, roundedQuantitySum#70328]
(744) Filter
Input [9]: [product#69873, plant#69874, demandChannel#69875, demandStream#69876, considerVariance#69877, demandTimeBuckets#69878, demandPointInTimeStart#69879, demandPointInTimeEnd#69880, demandPointInTime#69881]
Condition : (isnotnull(product#69873) AND isnotnull(plant#69874))
(745) BroadcastExchange
Input [9]: [product#69873, plant#69874, demandChannel#69875, demandStream#69876, considerVariance#69877, demandTimeBuckets#69878, demandPointInTimeStart#69879, demandPointInTimeEnd#69880, demandPointInTime#69881]
Arguments: HashedRelationBroadcastMode(List(input[0, string, false], input[1, string, false]),false), [plan_id=32615]
(746) BroadcastHashJoin
Left keys [2]: [product#69978, plant#69979]
Right keys [2]: [product#69873, plant#69874]
Join type: LeftOuter
Join condition: None
(747) Project
Output [9]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, CASE WHEN isnotnull(product#69873) THEN struct(demandChannel, demandChannel#69875, demandStream, demandStream#69876, considerVariance, considerVariance#69877, demandTimeBuckets, demandTimeBuckets#69878, demandPointInTimeStart, demandPointInTimeStart#69879, demandPointInTimeEnd, demandPointInTimeEnd#69880, demandPointInTime, demandPointInTime#69881) END AS rawDemands#69908]
Input [17]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, product#69873, plant#69874, demandChannel#69875, demandStream#69876, considerVariance#69877, demandTimeBuckets#69878, demandPointInTimeStart#69879, demandPointInTimeEnd#69880, demandPointInTime#69881]
(748) ObjectHashAggregate
Input [9]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, rawDemands#69908]
Keys [9]: [product#69978, plant#69979, orderDateTime#69980, knownfloatingpointnormalized(if (isnull(productPlant#69990)) null else named_struct(systemId, productPlant#69990.systemId, internalUUID, productPlant#69990.internalUUID, plant, productPlant#69990.plant, product, productPlant#69990.product, blockedForReplenishmentStartingFrom, productPlant#69990.blockedForReplenishmentStartingFrom, productionAspect, knownfloatingpointnormalized(if (isnull(productPlant#69990.productionAspect)) null else named_struct(productMovementPlants, productPlant#69990.productionAspect.productMovementPlants, productPlanningPlants, knownfloatingpointnormalized(if (isnull(productPlant#69990.productionAspect.productPlanningPlants)) null else named_struct(mrpType, productPlant#69990.productionAspect.productPlanningPlants.mrpType, abcIndicator, productPlant#69990.productionAspect.productPlanningPlants.abcIndicator, mrpController, productPlant#69990.productionAspect.productPlanningPlants.mrpController, minimumLotSize, knownfloatingpointnormalized(normalizenanandzero(productPlant#69990.productionAspect.productPlanningPlants.minimumLotSize)), minimumSafetyStock, knownfloatingpointnormalized(normalizenanandzero(productPlant#69990.productionAspect.productPlanningPlants.minimumSafetyStock)), logisticalRoundingProfile, productPlant#69990.productionAspect.productPlanningPlants.logisticalRoundingProfile, sourceOfSupplyCategory, productPlant#69990.productionAspect.productPlanningPlants.sourceOfSupplyCategory)))), salesPlant, productPlant#69990.salesPlant, listing, productPlant#69990.listing)) AS productPlant#69990, timezonecode#70005, knownfloatingpointnormalized(normalizenanandzero(outboundUnit#70022)) AS outboundUnit#70022, stockDatetime#70329, knownfloatingpointnormalized(transform(stockBatchQuantities#70330, lambdafunction(knownfloatingpointnormalized(normalizenanandzero(lambda arg#73953)), lambda arg#73953, false))) AS stockBatchQuantities#70330, [7226582400000000] AS [7226582400000000]#73952]
Functions [1]: [partial_collect_list(rawDemands#69908, 0, 0)]
Aggregate Attributes [1]: [buf#73954]
Results [10]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, [7226582400000000]#73952, buf#73955]
(749) ObjectHashAggregate
Input [10]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, [7226582400000000]#73952, buf#73955]
Keys [9]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, [7226582400000000]#73952]
Functions [1]: [collect_list(rawDemands#69908, 0, 0)]
Aggregate Attributes [1]: [collect_list(rawDemands#69908, 0, 0)#69939]
Results [10]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, [7226582400000000] AS stockBatchShelfLifeEndDatetimes#70331, collect_list(rawDemands#69908, 0, 0)#69939 AS rawDemands#69921]
(750) DeserializeToObject
Input [10]: [product#69978, plant#69979, orderDateTime#69980, productPlant#69990, timezonecode#70005, outboundUnit#70022, stockDatetime#70329, stockBatchQuantities#70330, stockBatchShelfLifeEndDatetimes#70331, rawDemands#69921]
Arguments: createexternalrow(invoke(product#69978.toString()), invoke(plant#69979.toString()), static_invoke(DateTimeUtils.toJavaTimestamp(orderDateTime#69980)), if (isnull(productPlant#69990)) null else createexternalrow(invoke(productPlant#69990.systemId.toString()), invoke(productPlant#69990.internalUUID.toString()), if (isnull(productPlant#69990.plant)) null else createexternalrow(invoke(productPlant#69990.plant.internalRefUUID.toString()), invoke(productPlant#69990.plant.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), if (isnull(productPlant#69990.product)) null else createexternalrow(invoke(productPlant#69990.product.internalRefUUID.toString()), invoke(productPlant#69990.product.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(DateTimeUtils.toJavaDate(productPlant#69990.blockedForReplenishmentStartingFrom)), if (isnull(productPlant#69990.productionAspect)) null else createexternalrow(if (isnull(productPlant#69990.productionAspect.productMovementPlants)) null else createexternalrow(if (isnull(productPlant#69990.productionAspect.productMovementPlants.unitOfIssue)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productMovementPlants.unitOfIssue.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productMovementPlants.unitOfIssue.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), if (isnull(productPlant#69990.productionAspect.productPlanningPlants)) null else createexternalrow(if (isnull(productPlant#69990.productionAspect.productPlanningPlants.mrpType)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productPlanningPlants.mrpType.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productPlanningPlants.mrpType.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#69990.productionAspect.productPlanningPlants.abcIndicator)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productPlanningPlants.abcIndicator.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productPlanningPlants.abcIndicator.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), if (isnull(productPlant#69990.productionAspect.productPlanningPlants.mrpController)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productPlanningPlants.mrpController.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productPlanningPlants.mrpController.id.toString()), StructField(internalRefUUID,StringType,true), StructField(id,StringType,true)), static_invoke(java.lang.Double.valueOf(productPlant#69990.productionAspect.productPlanningPlants.minimumLotSize)), static_invoke(java.lang.Double.valueOf(productPlant#69990.productionAspect.productPlanningPlants.minimumSafetyStock)), if (isnull(productPlant#69990.productionAspect.productPlanningPlants.logisticalRoundingProfile)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productPlanningPlants.logisticalRoundingProfile.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productPlanningPlants.logisticalRoundingProfile.logisticalRoundingProfile.toString()), StructField(internalRefUUID,StringType,true), StructField(logisticalRoundingProfile,StringType,true)), if (isnull(productPlant#69990.productionAspect.productPlanningPlants.sourceOfSupplyCategory)) null else createexternalrow(invoke(productPlant#69990.productionAspect.productPlanningPlants.sourceOfSupplyCategory.internalRefUUID.toString()), invoke(productPlant#69990.productionAspect.productPlanningPlants.sourceOfSupplyCategory.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true), StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(minimumLotSize,DoubleType,true), StructField(minimumSafetyStock,DoubleType,true), StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true), StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)), if (isnull(productPlant#69990.salesPlant)) null else createexternalrow(if (isnull(productPlant#69990.salesPlant.loadingGroup)) null else createexternalrow(invoke(productPlant#69990.salesPlant.loadingGroup.internalRefUUID.toString()), invoke(productPlant#69990.salesPlant.loadingGroup.code.toString()), StructField(internalRefUUID,StringType,true), StructField(code,StringType,true)), StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)), mapobjects(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1), if (isnull(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validFrom)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).validTo)), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true), true, -1).isListed)), StructField(validFrom,TimestampType,true), StructField(validTo,TimestampType,true), StructField(isListed,BooleanType,true)), productPlant#69990.listing, Some(class scala.collection.mutable.ArraySeq)), StructField(systemId,StringType,true), StructField(internalUUID,StringType,true), StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true), StructField(blockedForReplenishmentStartingFrom,DateType,true), StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true), StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true), StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)), invoke(timezonecode#70005.toString()), static_invoke(java.lang.Double.valueOf(outboundUnit#70022)), static_invoke(DateTimeUtils.toJavaTimestamp(stockDatetime#70329)), mapobjects(lambdavariable(MapObject, DoubleType, true, -2), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, DoubleType, true, -2))), stockBatchQuantities#70330, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, TimestampType, false, -3), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, TimestampType, false, -3))), stockBatchShelfLifeEndDatetimes#70331, Some(class scala.collection.mutable.ArraySeq)), mapobjects(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4), createexternalrow(invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandChannel.toString()), invoke(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandStream.toString()), static_invoke(java.lang.Boolean.valueOf(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).considerVariance)), mapobjects(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5), if (isnull(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandStartDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -5).demandVariance)), StructField(demandStartDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandTimeBuckets, Some(class scala.collection.mutable.ArraySeq)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeStart)), static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTimeEnd)), mapobjects(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6), if (isnull(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6))) null else createexternalrow(static_invoke(DateTimeUtils.toJavaTimestamp(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandDateTime)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandQuantity)), static_invoke(java.lang.Double.valueOf(lambdavariable(MapObject, StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true), true, -6).demandVariance)), StructField(demandDateTime,TimestampType,true), StructField(demandQuantity,DoubleType,true), StructField(demandVariance,DoubleType,true)), lambdavariable(MapObject, StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), false, -4).demandPointInTime, Some(class scala.collection.mutable.ArraySeq)), StructField(demandChannel,StringType,true), StructField(demandStream,StringType,true), StructField(considerVariance,BooleanType,true), StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true), StructField(demandPointInTimeStart,TimestampType,true), StructField(demandPointInTimeEnd,TimestampType,true), StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)), rawDemands#69921, Some(class scala.collection.mutable.ArraySeq)), StructField(product,StringType,true), StructField(plant,StringType,true), StructField(orderDateTime,TimestampType,true), StructField(productPlant,StructType(StructField(systemId,StringType,true),StructField(internalUUID,StringType,true),StructField(plant,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(product,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(blockedForReplenishmentStartingFrom,DateType,true),StructField(productionAspect,StructType(StructField(productMovementPlants,StructType(StructField(unitOfIssue,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(productPlanningPlants,StructType(StructField(mrpType,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(abcIndicator,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true),StructField(mrpController,StructType(StructField(internalRefUUID,StringType,true),StructField(id,StringType,true)),true),StructField(minimumLotSize,DoubleType,true),StructField(minimumSafetyStock,DoubleType,true),StructField(logisticalRoundingProfile,StructType(StructField(internalRefUUID,StringType,true),StructField(logisticalRoundingProfile,StringType,true)),true),StructField(sourceOfSupplyCategory,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true)),true),StructField(salesPlant,StructType(StructField(loadingGroup,StructType(StructField(internalRefUUID,StringType,true),StructField(code,StringType,true)),true)),true),StructField(listing,ArrayType(StructType(StructField(validFrom,TimestampType,true),StructField(validTo,TimestampType,true),StructField(isListed,BooleanType,true)),true),true)),true), StructField(timezonecode,StringType,true), StructField(outboundUnit,DoubleType,true), StructField(stockDatetime,TimestampType,true), StructField(stockBatchQuantities,ArrayType(DoubleType,true),false), StructField(stockBatchShelfLifeEndDatetimes,ArrayType(TimestampType,false),false), StructField(rawDemands,ArrayType(StructType(StructField(demandChannel,StringType,true),StructField(demandStream,StringType,true),StructField(considerVariance,BooleanType,true),StructField(demandTimeBuckets,ArrayType(StructType(StructField(demandStartDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true),StructField(demandPointInTimeStart,TimestampType,true),StructField(demandPointInTimeEnd,TimestampType,true),StructField(demandPointInTime,ArrayType(StructType(StructField(demandDateTime,TimestampType,true),StructField(demandQuantity,DoubleType,true),StructField(demandVariance,DoubleType,true)),true),true)),false),false)), obj#69963: org.apache.spark.sql.Row
(751) MapElements
Input [1]: [obj#69963]
Arguments: com.sap.s4hana.eureka.business.crporderquantitykpiservice.core.business.datasnapshot.DemandTransformatorImpl$$Lambda/0x00000008024b0d88@783ec352, obj#69968: org.apache.spark.sql.Row
(752) SerializeFromObject
Input [1]: [obj#69968]
Arguments: [if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(0))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 0, product), StringType, ObjectType(class java.lang.String)))) AS product#69973, if (invoke(assertnotnull(input[0, org.apache.spark.sql.Row, true]).isNullAt(1))) null else static_invoke(UTF8String.fromString(validateexternaltype(getexternalrowfield(assertnotnull(input[0, org.apache.spark.sql.Row, true]), 1, plant), StringType, ObjectType(class java.lang.String)))) AS plant#69974]
(753) Exchange
Input [2]: [product#69973, plant#69974]
Arguments: hashpartitioning(coalesce(plant#69974, ), isnull(plant#69974), coalesce(product#69973, ), isnull(product#69973), 25), ENSURE_REQUIREMENTS, [plan_id=32633]
(754) Sort
Input [2]: [product#69973, plant#69974]
Arguments: [coalesce(plant#69974, ) ASC NULLS FIRST, isnull(plant#69974) ASC NULLS FIRST, coalesce(product#69973, ) ASC NULLS FIRST, isnull(product#69973) ASC NULLS FIRST], false, 0
(755) SortMergeJoin
Left keys [4]: [coalesce(plant#69020, ), isnull(plant#69020), coalesce(product#69019, ), isnull(product#69019)]
Right keys [4]: [coalesce(plant#69974, ), isnull(plant#69974), coalesce(product#69973, ), isnull(product#69973)]
Join type: LeftOuter
Join condition: None
(756) Project
Output [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Input [6]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868, product#69973, plant#69974]
(757) Exchange
Input [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: hashpartitioning(plant#69020, 25), ENSURE_REQUIREMENTS, [plan_id=32697]
(758) Sort
Input [4]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868]
Arguments: [plant#69020 ASC NULLS FIRST], false, 0
(759) Scan parquet
Output [2]: [marketUnit#70809, distributionCenters#70815]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/prp/market-unit]
PushedFilters: [EqualNullSafe(marketUnit,Daily_grocery), IsNotNull(distributionCenters)]
ReadSchema: struct<marketUnit:string,distributionCenters:array<struct<internalRefUUID:string,id:string,storageLocations:array<struct<internalRefUUID:string,id:string>>>>>
(760) Filter
Input [2]: [marketUnit#70809, distributionCenters#70815]
Condition : (((marketUnit#70809 <=> Daily_grocery) AND (size(distributionCenters#70815, true) > 0)) AND isnotnull(distributionCenters#70815))
(761) Project
Output [1]: [distributionCenters#70815]
Input [2]: [marketUnit#70809, distributionCenters#70815]
(762) Exchange
Input [1]: [distributionCenters#70815]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31577]
(763) Generate
Input [1]: [distributionCenters#70815]
Arguments: explode(distributionCenters#70815), false, [distributionCenters#70854]
(764) Filter
Input [1]: [distributionCenters#70854]
Condition : (((size(distributionCenters#70854.storageLocations.internalRefUUID, true) > 0) AND isnotnull(distributionCenters#70854.storageLocations.internalRefUUID)) AND isnotnull(distributionCenters#70854.internalRefUUID))
(765) Project
Output [2]: [distributionCenters#70854.internalRefUUID AS _extract_internalRefUUID#71061, distributionCenters#70854.storageLocations.internalRefUUID AS _extract_internalRefUUID#71429]
Input [1]: [distributionCenters#70854]
(766) Generate
Input [2]: [_extract_internalRefUUID#71061, _extract_internalRefUUID#71429]
Arguments: explode(_extract_internalRefUUID#71429), [_extract_internalRefUUID#71061], false, [storageLocations#70855]
(767) Filter
Input [2]: [_extract_internalRefUUID#71061, storageLocations#70855]
Condition : isnotnull(storageLocations#70855)
(768) Project
Output [2]: [_extract_internalRefUUID#71061 AS plantUuid#70858, storageLocations#70855 AS storageLocUuid#70862]
Input [2]: [_extract_internalRefUUID#71061, storageLocations#70855]
(769) BroadcastExchange
Input [2]: [plantUuid#70858, storageLocUuid#70862]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32683]
(770) Scan parquet
Output [2]: [product#70865, plant#70866]
Batched: true
Location: InMemoryFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-order-qty-opt-service/out/product-plant-list/10000002106/shardId=0_1_10000002106]
PushedFilters: [IsNotNull(plant)]
ReadSchema: struct<product:string,plant:string>
(771) Filter
Input [2]: [product#70865, plant#70866]
Condition : isnotnull(plant#70866)
(772) Exchange
Input [2]: [product#70865, plant#70866]
Arguments: hashpartitioning(product#70865, plant#70866, 25), REPARTITION_BY_NUM, [plan_id=31971]
(773) Scan parquet
Output [4]: [plant#70883, product#70884, productionAspect#70886, systemId#70881]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-productplant]
ReadSchema: struct<plant:struct<internalRefUUID:string>,product:struct<internalRefUUID:string>,productionAspect:struct<productMovementPlants:struct<unitOfIssue:struct<internalRefUUID:string,code:string>>>>
(774) Project
Output [4]: [plant#70883.internalRefUUID AS _extract_internalRefUUID#71431, product#70884.internalRefUUID AS _extract_internalRefUUID#71434, productionAspect#70886.productMovementPlants.unitOfIssue.code AS _extract_code#71432, productionAspect#70886.productMovementPlants.unitOfIssue.internalRefUUID AS _extract_internalRefUUID#71433]
Input [4]: [plant#70883, product#70884, productionAspect#70886, systemId#70881]
(775) Exchange
Input [4]: [_extract_internalRefUUID#71431, _extract_internalRefUUID#71434, _extract_code#71432, _extract_internalRefUUID#71433]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31590]
(776) Project
Output [4]: [_extract_code#71432, _extract_internalRefUUID#71433, _extract_internalRefUUID#71434, _extract_internalRefUUID#71431]
Input [4]: [_extract_internalRefUUID#71431, _extract_internalRefUUID#71434, _extract_code#71432, _extract_internalRefUUID#71433]
(777) BroadcastExchange
Input [4]: [_extract_code#71432, _extract_internalRefUUID#71433, _extract_internalRefUUID#71434, _extract_internalRefUUID#71431]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[2, string, true], ), isnull(input[2, string, true]), coalesce(input[3, string, true], ), isnull(input[3, string, true])),false), [plan_id=32650]
(778) BroadcastHashJoin
Left keys [4]: [coalesce(product#70865, ), isnull(product#70865), coalesce(plant#70866, ), isnull(plant#70866)]
Right keys [4]: [coalesce(_extract_internalRefUUID#71434, ), isnull(_extract_internalRefUUID#71434), coalesce(_extract_internalRefUUID#71431, ), isnull(_extract_internalRefUUID#71431)]
Join type: LeftOuter
Join condition: None
(779) Project
Output [4]: [product#70865, plant#70866, _extract_code#71432, _extract_internalRefUUID#71433]
Input [6]: [product#70865, plant#70866, _extract_code#71432, _extract_internalRefUUID#71433, _extract_internalRefUUID#71434, _extract_internalRefUUID#71431]
(780) Scan parquet
Output [2]: [internalUUID#70892, systemId#70891]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-plant-plant]
ReadSchema: struct<internalUUID:string>
(781) Project
Output [1]: [internalUUID#70892]
Input [2]: [internalUUID#70892, systemId#70891]
(782) Exchange
Input [1]: [internalUUID#70892]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31596]
(783) BroadcastExchange
Input [1]: [internalUUID#70892]
Arguments: HashedRelationBroadcastMode(List(coalesce(input[0, string, true], ), isnull(input[0, string, true])),false), [plan_id=32657]
(784) BroadcastHashJoin
Left keys [2]: [coalesce(plant#70866, ), isnull(plant#70866)]
Right keys [2]: [coalesce(internalUUID#70892, ), isnull(internalUUID#70892)]
Join type: LeftOuter
Join condition: None
(785) Project
Output [4]: [product#70865, plant#70866, _extract_code#71432, _extract_internalRefUUID#71433]
Input [5]: [product#70865, plant#70866, _extract_code#71432, _extract_internalRefUUID#71433, internalUUID#70892]
(786) Scan parquet
Output [4]: [internalUUID#70919, baseUnitOfMeasure#70923, unitOfMeasureConversions#70925, systemId#70918]
Batched: true
Location: PreparedDeltaFileIndex [hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/crp-dl-stream-service/cornerstone/sap-cic-product-product-v2]
PushedFilters: [IsNotNull(internalUUID)]
ReadSchema: struct<internalUUID:string,baseUnitOfMeasure:struct<internalRefUUID:string,code:string>,unitOfMeasureConversions:array<struct<measurementUnit1:struct<internalRefUUID:string,code:string>,measurementUnit2:struct<internalRefUUID:string,code:string>,quantityNumerator:int,quantityDenominator:int>>>
(787) Filter
Input [4]: [internalUUID#70919, baseUnitOfMeasure#70923, unitOfMeasureConversions#70925, systemId#70918]
Condition : isnotnull(internalUUID#70919)
(788) Project
Output [3]: [internalUUID#70919, baseUnitOfMeasure#70923, unitOfMeasureConversions#70925]
Input [4]: [internalUUID#70919, baseUnitOfMeasure#70923, unitOfMeasureConversions#70925, systemId#70918]
(789) Exchange
Input [3]: [internalUUID#70919, baseUnitOfMeasure#70923, unitOfMeasureConversions#70925]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31602]
(790) BroadcastExchange
Input [3]: [internalUUID#70919, baseUnitOfMeasure#70923, unitOfMeasureConversions#70925]
Arguments: HashedRelationBroadcastMode(List(input[0, string, true]),false), [plan_id=32663]
(791) BroadcastHashJoin
Left keys [1]: [product#70865]
Right keys [1]: [internalUUID#70919]
Join type: LeftOuter
Join condition: None
(792) Project
Output [2]: [plant#70866, filter(unitOfMeasureConversions#70925, lambdafunction((((lambda u#69109.measurementUnit1.code = _extract_code#71432) AND (lambda u#69109.measurementUnit1.internalRefUUID = _extract_internalRefUUID#71433)) AND ((lambda u#69109.measurementUnit2.code = baseUnitOfMeasure#70923.code) AND (lambda u#69109.measurementUnit2.internalRefUUID = baseUnitOfMeasure#70923.internalRefUUID))), lambda u#69109, false)) AS filteredUnitOfMeasureConversions#70943]
Input [7]: [product#70865, plant#70866, _extract_code#71432, _extract_internalRefUUID#71433, internalUUID#70919, baseUnitOfMeasure#70923, unitOfMeasureConversions#70925]
(793) Generate
Input [2]: [plant#70866, filteredUnitOfMeasureConversions#70943]
Arguments: explode(filteredUnitOfMeasureConversions#70943.quantityNumerator), [plant#70866, filteredUnitOfMeasureConversions#70943], true, [quantityNumerator#70944]
(794) Project
Output [2]: [plant#70866, filteredUnitOfMeasureConversions#70943.quantityDenominator AS _extract_quantityDenominator#71064]
Input [3]: [plant#70866, filteredUnitOfMeasureConversions#70943, quantityNumerator#70944]
(795) Generate
Input [2]: [plant#70866, _extract_quantityDenominator#71064]
Arguments: explode(_extract_quantityDenominator#71064), [plant#70866], true, [quantityDenominator#70946]
(796) Project
Output [1]: [plant#70866]
Input [2]: [plant#70866, quantityDenominator#70946]
(797) HashAggregate
Input [1]: [plant#70866]
Keys [1]: [plant#70866]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70866]
(798) Exchange
Input [1]: [plant#70866]
Arguments: hashpartitioning(plant#70866, 25), ENSURE_REQUIREMENTS, [plan_id=32679]
(799) HashAggregate
Input [1]: [plant#70866]
Keys [1]: [plant#70866]
Functions: []
Aggregate Attributes: []
Results [1]: [plant#70866]
(800) BroadcastHashJoin
Left keys [1]: [plantUuid#70858]
Right keys [1]: [plant#70866]
Join type: Inner
Join condition: None
(801) Project
Output [2]: [plantUuid#70858 AS plant#70807, storageLocUuid#70862]
Input [3]: [plantUuid#70858, storageLocUuid#70862, plant#70866]
(802) Sort
Input [2]: [plant#70807, storageLocUuid#70862]
Arguments: [plant#70807 ASC NULLS FIRST], false, 0
(803) SortMergeJoin
Left keys [1]: [plant#69020]
Right keys [1]: [plant#70807]
Join type: Inner
Join condition: None
(804) Project
Output [5]: [plant#69020, product#69019, timezonecode#69079, stockDatetime#69868, storageLocUuid#70862]
Input [6]: [product#69019, plant#69020, timezonecode#69079, stockDatetime#69868, plant#70807, storageLocUuid#70862]
(805) Filter
Input [6]: [internalUUID#70730, id#70731, supplier#70733, isMarkedForDeletion#70736, items#70737, systemId#70729]
Condition : (((isnotnull(isMarkedForDeletion#70736) AND NOT isMarkedForDeletion#70736) AND (size(items#70737, true) > 0)) AND isnotnull(items#70737))
(806) Project
Output [4]: [internalUUID#70730, id#70731, supplier#70733, items#70737]
Input [6]: [internalUUID#70730, id#70731, supplier#70733, isMarkedForDeletion#70736, items#70737, systemId#70729]
(807) Exchange
Input [4]: [internalUUID#70730, id#70731, supplier#70733, items#70737]
Arguments: RoundRobinPartitioning(25), REPARTITION_BY_NUM, [plan_id=31618]
(808) Generate
Input [4]: [internalUUID#70730, id#70731, supplier#70733, items#70737]
Arguments: explode(items#70737), [internalUUID#70730, id#70731, supplier#70733], false, [item#70753]
(809) Filter
Input [4]: [internalUUID#70730, id#70731, supplier#70733, item#70753]
Condition : ((((isnotnull(item#70753.deletionIndicator) AND isnotnull(item#70753.completeIndicator)) AND isnotnull(item#70753.returnIndicator)) AND ((NOT item#70753.deletionIndicator AND NOT item#70753.completeIndicator) AND NOT item#70753.returnIndicator)) AND ((isnotnull(item#70753.product.internalRefUUID) AND isnotnull(item#70753.plant.internalRefUUID)) AND isnotnull(item#70753.storageLocation.internalRefUUID)))
(810) Project
Output [14]: [id#70731, internalUUID#70730, supplier#70733.internalRefUUID AS supplierUuid#70747, item#70753.plant.internalRefUUID AS plant#70770, item#70753.product.internalRefUUID AS product#70773, item#70753.plant.id AS externalPlantId#70775, item#70753.product.id AS externalProductId#70777, item#70753.storageLocation.internalRefUUID AS storageLocUuid#70778, item#70753.quantityUnit.code AS quantityUom#70779, item#70753.quantityUnit.internalRefUUID AS quantityUomUuid#70780, item#70753.quantityNumerator AS quantityNumerator#70781, item#70753.quantityDenominator AS quantityDenominator#70782, item#70753.scheduleLines AS scheduleLines#70783, item#70753.confirmations AS confirmations#70784]
Input [4]: [internalUUID#70730, id#70731, supplier#70733, item#70753]
(811) BroadcastExchange
Input [14]: [id#70731, internalUUID#70730, supplierUuid#70747, plant#70770, product#70773, externalPlantId#70775, externalProductId#70777, storageLocUuid#70778, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, confirmations#70784]
Arguments: HashedRelationBroadcastMode(List(input[4, string, true], input[3, string, true], input[7, string, true]),false), [plan_id=32703]
(812) BroadcastHashJoin
Left keys [3]: [product#69019, plant#69020, storageLocUuid#70862]
Right keys [3]: [product#70773, plant#70770, storageLocUuid#70778]
Join type: Inner
Join condition: None
(813) Project
Output [17]: [id#70731, internalUUID#70730, supplierUuid#70747, plant#70770, product#70773, externalPlantId#70775, externalProductId#70777, storageLocUuid#70778, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, confirmations#70784, stockDatetime#69868, timezonecode#69079, filter(confirmations#70784, lambdafunction(((NOT lambda x_2#70966.deletionIndicator AND lambda x_2#70966.isRelevantForConfirmation) AND isnotnull(lambda x_2#70966.confirmedDeliveryAt)), lambda x_2#70966, false)) AS validConfirmations#70963]
Input [19]: [plant#69020, product#69019, timezonecode#69079, stockDatetime#69868, storageLocUuid#70862, id#70731, internalUUID#70730, supplierUuid#70747, plant#70770, product#70773, externalPlantId#70775, externalProductId#70777, storageLocUuid#70778, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, confirmations#70784]
(814) AdaptiveSparkPlan
Output [17]: [id#70731, internalUUID#70730, supplierUuid#70747, plant#70770, product#70773, externalPlantId#70775, externalProductId#70777, storageLocUuid#70778, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, confirmations#70784, stockDatetime#69868, timezonecode#69079, validConfirmations#70963]
Arguments: isFinalPlan=true
(815) TableCacheQueryStage
Output [10]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityDenominator#70782, quantityNumerator#70781, quantityUom#70779, quantityUomUuid#70780, scheduleLines#70783, validConfirmations#70963]
Arguments: 0
(816) Project [codegen id : 1]
Output [11]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, size(validConfirmations#70963, false) AS _w0#75046]
Input [10]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityDenominator#70782, quantityNumerator#70781, quantityUom#70779, quantityUomUuid#70780, scheduleLines#70783, validConfirmations#70963]
(817) Exchange
Input [11]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, _w0#75046]
Arguments: hashpartitioning(internalUUID#70730, 25), ENSURE_REQUIREMENTS, [plan_id=129398]
(818) ShuffleQueryStage
Output [11]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, _w0#75046]
Arguments: 1
(819) AQEShuffleRead
Input [11]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, _w0#75046]
Arguments: coalesced
(820) Sort [codegen id : 2]
Input [11]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, _w0#75046]
Arguments: [internalUUID#70730 ASC NULLS FIRST], false, 0
(821) Window
Input [11]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, _w0#75046]
Arguments: [sum(_w0#75046) windowspecdefinition(internalUUID#70730, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#75049L], [internalUUID#70730]
(822) Project
Output [3]: [plant#70770, product#70773, filter(transform(CASE WHEN (_we0#75049L > 0) THEN transform(validConfirmations#70963, lambdafunction(struct(openQty, greatest((lambda x_16#75039.confirmedQuantity - coalesce(lambda x_16#75039.mrpRelevantQuantity, 0.0)), 0.0), availabilityDatetime, lambda x_16#75039.confirmedDeliveryAt), lambda x_16#75039, false)) ELSE transform(scheduleLines#70783, lambdafunction(struct(openQty, greatest((lambda x_15#75041.quantity - coalesce(lambda x_15#75041.receivedQuantity, 0.0)), 0.0), availabilityDatetime, lambda x_15#75041.deliveryAt), lambda x_15#75041, false)) END, lambdafunction(struct(documentId, id#70731, documentUuid, internalUUID#70730, openQty, lambda x_17#75043.openQty, openQtyInBaseUnit, ((lambda x_17#75043.openQty * quantityNumerator#70781) / quantityDenominator#70782), qtyUom, quantityUom#70779, qtyUomUuid, quantityUomUuid#70780, availabilityDatetime, lambda x_17#75043.availabilityDatetime, shelfLifeEndDatetime, 2199-01-01 00:00:00), lambda x_17#75043, false)), lambdafunction(NOT (lambda x_18#75045.openQty = 0.0), lambda x_18#75045, false)) AS openGoodsMovements#75035]
Input [12]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, _w0#75046, _we0#75049L]
(823) ObjectHashAggregate
Input [3]: [plant#70770, product#70773, openGoodsMovements#75035]
Keys [2]: [product#70773, plant#70770]
Functions [1]: [partial_collect_list(openGoodsMovements#75035, 0, 0)]
Aggregate Attributes [1]: [buf#75600]
Results [3]: [product#70773, plant#70770, buf#75601]
(824) Exchange
Input [3]: [product#70773, plant#70770, buf#75601]
Arguments: hashpartitioning(product#70773, plant#70770, 25), ENSURE_REQUIREMENTS, [plan_id=129457]
(825) ShuffleQueryStage
Output [3]: [product#70773, plant#70770, buf#75601]
Arguments: 2
(826) ObjectHashAggregate
Input [3]: [product#70773, plant#70770, buf#75601]
Keys [2]: [product#70773, plant#70770]
Functions [1]: [collect_list(openGoodsMovements#75035, 0, 0)]
Aggregate Attributes [1]: [collect_list(openGoodsMovements#75035, 0, 0)#75088]
Results [3]: [product#70773, plant#70770, flatten(collect_list(openGoodsMovements#75035, 0, 0)#75088) AS openGoodsMovements#75050]
(827) Filter [codegen id : 3]
Input [3]: [product#70773, plant#70770, openGoodsMovements#75050]
Condition : (size(openGoodsMovements#75050, false) > 0)
(828) ResultQueryStage
Output [3]: [product#70773, plant#70770, openGoodsMovements#75050]
Arguments: 3
(829) Project
Output [11]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, size(validConfirmations#70963, false) AS _w0#75046]
Input [10]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityDenominator#70782, quantityNumerator#70781, quantityUom#70779, quantityUomUuid#70780, scheduleLines#70783, validConfirmations#70963]
(830) Exchange
Input [11]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, _w0#75046]
Arguments: hashpartitioning(internalUUID#70730, 25), ENSURE_REQUIREMENTS, [plan_id=32757]
(831) Sort
Input [11]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, _w0#75046]
Arguments: [internalUUID#70730 ASC NULLS FIRST], false, 0
(832) Window
Input [11]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, _w0#75046]
Arguments: [sum(_w0#75046) windowspecdefinition(internalUUID#70730, specifiedwindowframe(RowFrame, unboundedpreceding$(), unboundedfollowing$())) AS _we0#75049L], [internalUUID#70730]
(833) Project
Output [3]: [plant#70770, product#70773, filter(transform(CASE WHEN (_we0#75049L > 0) THEN transform(validConfirmations#70963, lambdafunction(struct(openQty, greatest((lambda x_16#75039.confirmedQuantity - coalesce(lambda x_16#75039.mrpRelevantQuantity, 0.0)), 0.0), availabilityDatetime, lambda x_16#75039.confirmedDeliveryAt), lambda x_16#75039, false)) ELSE transform(scheduleLines#70783, lambdafunction(struct(openQty, greatest((lambda x_15#75041.quantity - coalesce(lambda x_15#75041.receivedQuantity, 0.0)), 0.0), availabilityDatetime, lambda x_15#75041.deliveryAt), lambda x_15#75041, false)) END, lambdafunction(struct(documentId, id#70731, documentUuid, internalUUID#70730, openQty, lambda x_17#75043.openQty, openQtyInBaseUnit, ((lambda x_17#75043.openQty * quantityNumerator#70781) / quantityDenominator#70782), qtyUom, quantityUom#70779, qtyUomUuid, quantityUomUuid#70780, availabilityDatetime, lambda x_17#75043.availabilityDatetime, shelfLifeEndDatetime, 2199-01-01 00:00:00), lambda x_17#75043, false)), lambdafunction(NOT (lambda x_18#75045.openQty = 0.0), lambda x_18#75045, false)) AS openGoodsMovements#75035]
Input [12]: [id#70731, internalUUID#70730, plant#70770, product#70773, quantityUom#70779, quantityUomUuid#70780, quantityNumerator#70781, quantityDenominator#70782, scheduleLines#70783, validConfirmations#70963, _w0#75046, _we0#75049L]
(834) ObjectHashAggregate
Input [3]: [plant#70770, product#70773, openGoodsMovements#75035]
Keys [2]: [product#70773, plant#70770]
Functions [1]: [partial_collect_list(openGoodsMovements#75035, 0, 0)]
Aggregate Attributes [1]: [buf#75600]
Results [3]: [product#70773, plant#70770, buf#75601]
(835) Exchange
Input [3]: [product#70773, plant#70770, buf#75601]
Arguments: hashpartitioning(product#70773, plant#70770, 25), ENSURE_REQUIREMENTS, [plan_id=32768]
(836) ObjectHashAggregate
Input [3]: [product#70773, plant#70770, buf#75601]
Keys [2]: [product#70773, plant#70770]
Functions [1]: [collect_list(openGoodsMovements#75035, 0, 0)]
Aggregate Attributes [1]: [collect_list(openGoodsMovements#75035, 0, 0)#75088]
Results [3]: [product#70773, plant#70770, flatten(collect_list(openGoodsMovements#75035, 0, 0)#75088) AS openGoodsMovements#75050]
(837) Filter
Input [3]: [product#70773, plant#70770, openGoodsMovements#75050]
Condition : (size(openGoodsMovements#75050, false) > 0)
(838) AdaptiveSparkPlan
Output [3]: [product#70773, plant#70770, openGoodsMovements#75050]
Arguments: isFinalPlan=true
(839) TableCacheQueryStage
Output [3]: [openGoodsMovements#75050, plant#70770, product#70773]
Arguments: 0
(840) Filter
Input [3]: [openGoodsMovements#75050, plant#70770, product#70773]
Condition : (size(transform(openGoodsMovements#75050, lambdafunction(struct(documentType, PO, documentId, lambda x_27#90488.documentId, documentUuid, lambda x_27#90488.documentUuid, openQty, lambda x_27#90488.openQty, qtyUomUuid, lambda x_27#90488.qtyUomUuid, qtyUom, lambda x_27#90488.qtyUom, plannedDeliveryDatetime, lambda x_27#90488.availabilityDatetime), lambda x_27#90488, false)), false) > 0)
(841) Project
Output [4]: [product#70773, plant#70770 AS location#90484, transform(openGoodsMovements#75050, lambdafunction(struct(documentType, PO, documentId, lambda x_27#90488.documentId, documentUuid, lambda x_27#90488.documentUuid, openQty, lambda x_27#90488.openQty, qtyUomUuid, lambda x_27#90488.qtyUomUuid, qtyUom, lambda x_27#90488.qtyUom, plannedDeliveryDatetime, lambda x_27#90488.availabilityDatetime), lambda x_27#90488, false)) AS documents#90487, 10000002106 AS replenishmentRunId#90489L]
Input [3]: [openGoodsMovements#75050, plant#70770, product#70773]
(842) WriteFiles
Input [4]: [product#70773, location#90484, documents#90487, replenishmentRunId#90489L]
(843) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002106/0_1_10000002106, false, Parquet, [path=hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002106/0_1_10000002106], Append, [product, location, documents, replenishmentRunId]
(844) ResultQueryStage
Output: []
Arguments: 1
(845) Filter
Input [3]: [openGoodsMovements#75050, plant#70770, product#70773]
Condition : (size(transform(openGoodsMovements#75050, lambdafunction(struct(documentType, PO, documentId, lambda x_27#90488.documentId, documentUuid, lambda x_27#90488.documentUuid, openQty, lambda x_27#90488.openQty, qtyUomUuid, lambda x_27#90488.qtyUomUuid, qtyUom, lambda x_27#90488.qtyUom, plannedDeliveryDatetime, lambda x_27#90488.availabilityDatetime), lambda x_27#90488, false)), false) > 0)
(846) Project
Output [4]: [product#70773, plant#70770 AS location#90484, transform(openGoodsMovements#75050, lambdafunction(struct(documentType, PO, documentId, lambda x_27#90488.documentId, documentUuid, lambda x_27#90488.documentUuid, openQty, lambda x_27#90488.openQty, qtyUomUuid, lambda x_27#90488.qtyUomUuid, qtyUom, lambda x_27#90488.qtyUom, plannedDeliveryDatetime, lambda x_27#90488.availabilityDatetime), lambda x_27#90488, false)) AS documents#90487, 10000002106 AS replenishmentRunId#90489L]
Input [3]: [openGoodsMovements#75050, plant#70770, product#70773]
(847) WriteFiles
Input [4]: [product#70773, location#90484, documents#90487, replenishmentRunId#90489L]
(848) Execute InsertIntoHadoopFsRelationCommand
Input: []
Arguments: hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002106/0_1_10000002106, false, Parquet, [path=hdlfs://2e93940d-4be8-4f12-830d-f0b8d392c03a.files.hdl.prod-eu20.hanacloud.ondemand.com:443/prp-op-traceability-results/raw/crp-order-quantity-kpi-service/ogr/10000002106/0_1_10000002106], Append, [product, location, documents, replenishmentRunId]
(849) AdaptiveSparkPlan
Output: []
Arguments: isFinalPlan=true
| Name | Value |
|---|---|
| spark.activity.hdlfsEnabled | true |
| spark.activity.isolateSession | true |
| spark.databricks.delta.lastCommitVersionInSession | 0 |
| spark.sql.parquet.fieldId.read.enabled | true |
| spark.sql.parquet.fieldId.write.enabled | true |
| temporal.activity.id | 90595289-6f38-3cae-b43d-98e826007244 |
| temporal.activity.type | BufferDataSnapShot |