Constants.py

'''
https://huggingface.co/models
Most used:

bert-base-uncased
distilbert-base-uncased
roberta-base
google/electra-small-discriminator
YituTech/conv-bert-base
'''

# TRAIN_FILE_PATH = ['data/Col-STAT/debug.csv']
# TEST_FILE_PATH = ['data/Col-STAT/debug_test_set.csv']

TRAIN_FILE_PATH = ['./data/SemEval/5way/Sci_train.csv']
TEST_FILE_PATH = ['./data/SemEval/5way/Sci_test_UA.csv']

SEP_TOKEN = '[SEP]'
CLS_TOKEN = '[CLS]'

hyperparameters = dict(
    train_id="1024_SFRN_sci_5way_test0",
    model_name="bert-base-uncased",
    num_labels = 5,
    max_length = 128,
    random_seed=23, # 23， 
    data_split=0.2,
    lr=1e-5,
    epochs=23,
    weight_decay=0.01,
    GRADIENT_ACCUMULATION_STEPS=1,
    max_norm = 1, 
    WARMUP_STEPS=0.2,
    hidden_dropout_prob=0.2,
    # model
    hidden_dim=768, # 768
    mlp_hidden=128,
    )
# wandb config

config_dictionary = dict(
    #yaml=my_yaml_file,
    params=hyperparameters,
    )



q_text_dict = {
'ST_59':'Elena has a male lizard that has lived for several years in the habitat she provided. She knows that some lizards are territorial. Besides additional food and water, what should she be sure to include in the habitat before she adds another male lizard?',
'MX_41':'Janet found the solubility of silver chloride to be 86 grams in 50 milliliters of water. Mike found the solubility of silver nitrate to be 108 grams in 50 milliliters of water. Janet and Mike thought they must have made a mistake. They thought the solubilities of all materials in water should be the same. Is it possible that both solubilities are correct? Explain your answer.',
'FN_24c':'Diva\'s father told her she should not eat so many cookies because they were pure sugar. Diva decided to investigate the amount of sugar in Fruity Cream cookies. She performed the sugar test on 4 grams of pure sugar and on 4 grams of Fruity Cream cookies. The results are pictured at the right. Why did Diva need to test pure sugar?',
'SE_24a':'The drawing above shows a plaza in the afternoon. Label the west side of the drawing with a W. Label the north side of the drawing with an N. How did you know which direction was west?',
'FN_24b':'Diva\'s father told her she should not eat so many cookies because they were pure sugar. Diva decided to investigate the amount of sugar in Fruity Cream cookies. She performed the sugar test on 4 grams of pure sugar and on 4 grams of Fruity Cream cookies. The results are pictured at the right. Are Fruity Cream cookies pure sugar? What is your evidence?',
'MX_36b':'Grace set up a science experiment. Here\'s what she wrote in her notebook. First I put eggshells into a cup and covered them with vinegar. After a while, small bubbles appeared on the surface of the eggshells. The next day there were more bubbles. In a couple of days, the eggshells were gone and the liquid was clear. I think I made a solution and a reaction. Explain why Grace thinks she made a reaction.',
'SE_31b':'Julie and David each built a solar water heater. Both solar water heaters were 10 centimeters x 10 centimeters x 30 centimeters. Both water heaters were painted black on the inside bottom and filled to the top with 3 liters of room temperature water. Both water heaters were set in the sun. They checked the temperatures every 5 minutes and made the table below. Which box did Julie build, A or B? What evidence supports your answer?',
'LF_26b2':'Pam and her family were planning a hike. Pam found 2 maps of the same area. Her friend recommended she use the topographic map when they went to the lake. Which letter (Q, R, or S) on the map is on the steepest part of the trail? (Q) How do you know it is the steepest part of the trail?',
'VB_40a':'Jack made a lifeboat out of a large paper cup. He decided to measure how far the boat sank when different numbers of marbles were added. His data are shown in the table. Identify the standard for Jack\'s experiment.',
'MS_64a':'Greta used her hand to measure the length of the whiteboard in her classroom. She found that it was 14 hands long. Andre measured the whiteboard and found that it was 16 hands long. Explain why Greta and Andre got different measurements.',
'EM_46':'Describe how you can tell the difference between the crystals from the calcite test and salt crystals when you observe them in an evaporation dish.',
'ST_58':'What is your "rule" for deciding if the part of a plant you are observing is a fruit?',
'LF_31b':'What process forms deltas? A. Gravity B. Erosion (C. Deposition) D. Decomposition.  Explain how deltas form.',
'MX_36a':'Grace set up a science experiment. Here\'s what she wrote in her notebook. First I put eggshells into a cup and covered them with vinegar. After a while, small bubbles appeared on the surface of the eggshells. The next day there were more bubbles. In a couple of days, the eggshells were gone and the liquid was clear. I think I made a solution and a reaction. Explain why Grace thinks she made a solution.',
'PS_15bp':'Below are pictures of wind instruments. You blow air through them to produce sounds. Why does [the saxophone] make the lowest sounds?',
'MS_30b':'Mary told her friend Sharice that she had a rug to sell. Sharice asked if the rug would fit perfectly, wall to wall, in her bedroom. Mary went home and measured the rug. She put her feet heel-to-toe and counted the number of shoe lengths it took to measure the rug. She found the rug was 18 shoes long and 12 shoes wide. Sharice did the same thing in her room to measure the space for the rug. Her floor was 18 shoes long and 12 shoes wide. A perfect fit! Sharice bought the rug and took it home. To her dismay, she found it was too short and too narrow for her bedroom! What could Mary and Sharice do to better measure the size of the rug?',
'ME_78b':'Look at the picture of the circuit to the right. What will happen to the motor if the light bulb burns out?',
'PS_51a':'Katie got a guitar for her birthday. She experimented with the strings and found she could change their sounds. One way Katie could change the sound of a string was to tighten it. Describe how the sound was different when the string was tightened.',
'SE_22b':'Andi and Scott decided to investigate solar water heaters using collectors of different colors (but the same size) as one variable and covered versus uncovered as a second variable. Look at the graph of their data and answer the questions below. What does the graph tell you about the effect of using blue versus red collectors?',
'MS_43b':'2 groups used marbles to weigh the same apple. Jin\'s group found the apple weighed 35 marbles and Bee\'s group found the same apple weighed 60 marbles. Both groups used the correct procedure. What should the groups do to make their results more useful?',
'SE_45':'Betty was puzzled. In the morning the shadow of the bus sign is in the street. In the afternoon the shadow of the bus sign points the opposite direction. Explain to Betty why this happens.',
'MX_49':'Beth made 2 solutions of different concentrations of water and citric acid. She put 50 milliliters of the first solution in cup X and 50 milliliters of the second solution in cup Y. When she put the cups on the balance, the balance looked as shown below. Which solution, X or Y, is more concentrated? Explain how you know which is more concentrated.',
'ME_66a':'Look at the circuit in the picture. The gap between the D-cell and the switch needs to be connected. Can you use a piece of string to make the connection? Why or why not?',
'MX_22a':'A solution is a type of mixture. What makes it different from other mixtures?',
'SE_22c':'Andi and Scott decided to investigate solar water heaters using collectors of different colors (but the same size) as one variable and covered versus uncovered as a second variable. Look at the graph of their data and answer the questions below. Why would comparing only the red uncovered heater with the blue covered heater not provide useful information?',
'PS_51b':'Katie got a guitar for her birthday. She experimented with the strings and found she could change their sounds. Katie also discovered that when she pressed down on a string with her finger, making the string shorter, the sound again changed. Describe how the sound was different when the string was shortened.',
'SE_22a':'Andi and Scott decided to investigate solar water heaters using collectors of different colors (but the same size) as one variable and covered versus uncovered as a second variable. Look at the graph of their data and answer the questions below. What does the graph tell you about the effect of using a cover?',
'ME_5b':'Anne is investigating which objects stick to magnets. She recorded this in her notebook. I was surprised! A nail was stuck to the magnet. When I accidentally touched the nail to a paperclip, the paperclip stuck to the nail. If Anne touched the nail to a brass ring, what do you think would happen and why?',
'EM_47':'The sand and flour in the gray material from mock rocks is separated by mixing with water and allowing the mixture to settle. Explain why the sand and flour separate.',
'MS_43a':'2 groups used marbles to weigh the same apple. Jin\'s group found the apple weighed 35 marbles and Bee\'s group found the same apple weighed 60 marbles. Both groups used the correct procedure. Since they weighed the same apple, what do you think happened to make the results different?',
'SE_46':'Amber predicted that a black covered box would heat up faster than a white covered box. She took 2 identical shoe boxes and wrapped one with white paper and the other with black paper. She punched a hole in each box, stuck a thermometer into each hole, and put both boxes in the sun. Every 5 minutes, Amber recorded the temperature in each box. Will the data support Amber\'s prediction? Explain your answer.',
'II_20b':'When Sarah did the chromatography test, she saw that one black ink stayed black and didn\'t separate. She discovered that she had used a permanent marker. She wondered what would happen if she used a different liquid. She wrote in her science notebook, What if I used vinegar in place of the water? Can I get the pigments in permanent ink to separate? How will Sarah know if vinegar separated the permanent ink?',
'ME_66b':'Look at the circuit in the picture. The gap between the D-cell and the switch needs to be connected. Can you use a piece of aluminum foil to make the connection? Why or why not?',
'MS_50a':'Jen was talking on the phone with her Uncle Bill who lives in Canada. When she asked him how the weather was, he said it was nice and warm, about 28 degrees. What unit is Uncle Bill using to report the temperature?',
'ME_7a':'Lee has an object he wants to test to see if it is an insulator or a conductor. He is going to use the circuit you see in the picture. Explain how he can use the circuit to test the object.',
'SE_16b2':'Cory and Rachael set up 3 solar water heaters to see which would show the greatest water temperature change. The containers are shown below. Each is the same height (4 centimeters), and the same width (10 centimeters). Only the lengths are different; Container A is 10 centimeters, Container B is 30 centimeters, and Container C is 40 centimeters. They painted the plastic containers black, filled each with the same amount of water, and put a clear plastic lid on each one. Then they set them out in the sunshine. Here are the data that Cory and Rachael collected: After 10 minutes Container A was 3 degrees Celsius, Container B was 9 degrees Celsius, and Container C was 12 degrees Celsius. Use the graph paper on the next page to graph the temperature change for each container. If Cory and Rachael had a container that was the same height and width as the other containers, but 20 centimeters long, what do you think the temperature change would be after 10 minutes? (6 Celsius) How did you decide what the temperature change would be for the new container?',
'MX_11e':'Jim used a solid and water to make Mixtures one (one spoon of solid in 100 milliliters water was clear with nothing on the bottom), 3 (3 spoons of solid in 100 milliliters water was clear with nothing on the bottom), 4 (4 spoons of solid in 100 milliliters water was clear with material on the bottom), and 5 (5 spoons of solid in 100 milliliters water was clear with material on the bottom) as shown below. He stirred each one and observed the results. Describe how Jim could separate the dissolved solid from the water in Mixture one. What happens to the solid and to the water?',
'EM_35':'Carrie wanted to find out which was harder, a penny or a nickel, so she did a scratch test. How would this tell her which is harder?',
'LP_16d':'Look at the illustration of the pulley system at the right to answer the questions. What additional disadvantages might there be to this system compared to a single fixed pulley system?',
'ME_17b':'Denise made a circuit to light a bulb or run a motor off a D-cell battery. She used a special switch. Below is the schematic diagram of her circuit. The switch is inside the dotted box. Why will the bulb light when she moves the switch to the left?',
'EM_21a':'Georgia found one brown mineral and one black mineral. How can she test them for hardness using only the 2 minerals (no tools)?',
'MX_52b':'Emily put 20 milliliters of a mystery solution in an evaporation dish. When all the water was gone, she drew the picture of the dish you see here.  Using the crystal-shape key below, what chemical do you think was dissolved in the solution? A. Epsom salt B. Sodium thiosulfate (C. Alum) D. None of the above. Explain why you chose that answer.',
'ME_17c':'Denise made a circuit to light a bulb or run a motor. She used a special switch. Below is the schematic diagram of her circuit. The switch is inside the dotted box. What will happen when she moves the switch to the right?',
'PS_44':'A harp has strings of different lengths. Describe how the sound of a short string being plucked (pulled and let go quickly) is different from the sound of a long string being plucked.',
'SE_3c':'Why does the tree shown in the picture have a shadow?',
'PS_12':'Mary was trying to understand how tuning forks make sounds. She hit the tuning fork on the table and placed the tuning fork into a cup full of water while it was still ringing. The water splashed in all directions. How did seeing what happened help Mary understand how a tuning fork makes sounds?',
'ME_7b':'Lee has an object he wants to test to see if it is an insulator or a conductor. He is going to use the circuit you see in the picture. How will he know if the object is a conductor?',
'MX_11f':'Jim used a solid and water to make Mixtures one (one spoon of solid in 100 milliliters water was clear with nothing on the bottom), 3 (3 spoons of solid in 100 milliliters water was clear with nothing on the bottom), 4 (4 spoons of solid in 100 milliliters water was clear with material on the bottom), and 5 (5 spoons of solid in 100 milliliters water was clear with material on the bottom) as shown below. He stirred each one and observed the results. If Jim made Mixture 2 with 2 spoons of solid in 100 milliliters of water, what would he observe? What evidence do you have to support this?',
'ME_17a':'Denise made a circuit to light a bulb or run a motor. She used a special switch. Below is the schematic diagram of her circuit. The switch is inside the dotted box. What will happen when she moves the switch to the left?',
'EM_21b':'Georgia found one brown mineral and one black mineral. How will she know which one is harder?',
'VB_22c':'Describe what the graph tells you about the relationship between the number of scoops of fertilizer and the mass of beans harvested?',
'MX_42a':'Joanne mixed material R and material D together, added water, and stirred. After a few minutes, the liquid was clear and there was a solid on the bottom of the cup. Joanne used a paper filter to separate the solid from the liquid. Why does the solid stay in the filter when the liquid goes through?',
'ME_30':'Sometimes a red stoplight changes to green only when a car stops at the light. A type of magnet hidden under the surface of the road triggers the light to turn green. What is it about the car that could trigger a magnet to send a signal to the light?',
'LF_13a':'What happens to earth materials during deposition?',
'ME_65a':'Look at the schematic diagram. What will happen to the other 2 bulbs if the middle bulb burns out?',
'ME_69b':'A magnet attracted another magnet even when a foam block was placed between them (see box A below). What do you think would happen if 2 foam blocks were placed between the 2 magnets (see box B). A. The magnets will stick. B. The magnets will not stick. (C. It might stick, but it might not.) Explain why you chose that answer.',
'LP_15c':'What is the relationship between the amount of effort required to lift the load and its distance from the fulcrum?',
'MX_11c':'Jim used a solid and water to make Mixtures one (one spoon of solid in 100 milliliters water was clear with nothing on the bottom), 3 (3 spoons of solid in 100 milliliters water was clear with nothing on the bottom), 4 (4 spoons of solid in 100 milliliters water was clear with material on the bottom), and 5 (5 spoons of solid in 100 milliliters water was clear with material on the bottom) as shown below. He stirred each one and observed the results. Is the concentration of Mixture 5 less than, greater than, or the same as the concentration of Mixture 4? Explain your answer.',
'ME_17d':' Denise made a circuit to light a bulb or run a motor off a D-cell battery. She used a special switch. Below is the schematic diagram of her circuit. The switch is inside the dotted box. Why will the motor run when she moves the switch to the right?',
'MX_18':'A solution of water and material A is clear. A solution of water and material B is also clear. When the 2 solutions are mixed, a white solid settles to the bottom, leaving a clear liquid. What does the white solid indicate?',
'ME_10':'The motors shown at the right with the D-cell batteries will NOT run. Explain why.',
'VB_15c':'3 girls wanted to see who could throw a ball the farthest. Each girl brought her favorite ball to the park. Ling brought her basketball, Diana brought her football, and Rachael brought her baseball. They made a line on the playground. They all had to throw the ball from that line. Ling took 2 steps up to the line and threw her basketball. Diana stood at the line and threw her football. Rachael got a running start and threw her baseball when she reached the line. Rachael threw her ball the farthest. Describe what the girls need to do to make the contest a controlled experiment (fair test).',
'VB_15b':'3 girls wanted to see who could throw a ball the farthest. Each girl brought her favorite ball to the park. Ling brought her basketball, Diana brought her football, and Rachael brought her baseball. They made a line on the playground. They all had to throw the ball from that line. Ling took 2 steps up to the line and threw her basketball. Diana stood at the line and threw her football. Rachael got a running start and threw her baseball when she reached the line. Rachael threw her ball the farthest. Describe 2 variables that are not controlled in the contest.',
'MX_19':'When Amy added water to a mixture of baking soda and citric acid, the mixture began fizzing. Amy concluded there was a chemical reaction because it fizzed. Why is fizzing evidence of a chemical reaction?',
'ME_17e':'Denise made a circuit to light a bulb or run a motor. She used a special switch. Below is the schematic diagram of her circuit. The switch is inside the dotted box. Can she run both the light and the motor at the same time? Why or why not?',
'ME_27b':'How can you use a magnet to find out if the key is iron or aluminum?',
'EM_26':'Salt crystals form when the water evaporates from saltwater. Describe the salt crystals.',
'FN_19b':'Susan has samples of 5 different foods. Using only the results of her experiment, how will Susan know which food contains the most sugar?',
'ME_6b':'Look at the diagram of a circuit at the right. Will the motor run? Why?',
'ME_72':'Only one circuit below will light the bulb. Which one will work? Why is that the only circuit that will work?',
'EM_16b':'Ms. Teridann, a geologist, made a chart showing the scratch-test data for 3 minerals (Mineral X, Mineral Y, and Mineral Z). She wrote "yes" if the tool scratched the mineral and wrote "no" if it did not scratch it. Topaz is a mineral that cannot be scratched by a fingernail, a penny, or a paperclip. How hard is topaz compared to minerals X, Y, and Z? Explain how you know.',
'ME_65b':'Look at the schematic diagram including a D-cell battery and 3 bulbs. Why do the other 2 bulbs stay lit if the middle bulb burns out?',
'VB_15a':'3 girls wanted to see who could throw a ball the farthest. Each girl brought her favorite ball to the park. Ling brought her basketball, Diana brought her football, and Rachael brought her baseball. They made a line on the playground. They all had to throw the ball from that line. Ling took 2 steps up to the line and threw her basketball. Diana stood at the line and threw her football. Rachael got a running start and threw her baseball when she reached the line. Rachael threw her ball the farthest. What variable did the girls control in the contest?',
'MX_11a':'Jim used a solid and water to make Mixture one (one spoon of solid in 100 milliliters water was clear with nothing on the bottom), Mixture 3 (3 spoons of solid in 100 milliliters water was clear with nothing on the bottom), Mixture 4 (4 spoons of solid in 100 milliliters water was clear with material on the bottom), and Mixture 5 (5 spoons of solid in 100 milliliters water was clear with material on the bottom) as shown below. He stirred each one and observed the results. What evidence does Jim have that the solid and water make a solution?',
'LF_28a2':'Use the "slope" and "no slope" stream-table maps and logs on the facing page to answer these questions. What information do you get from the stream-table logs that you do not get from the stream-table maps?',
'PS_46b':'Why was the siren\'s sound designed to have this property?',
'ST_31b':'One function of the bess beetle\'s elytra (the hard, black wing set) is protection. What is another function of the elytra?',
'ME_28b':'Look at the picture on the right. Label the poles on each magnet. (The bottom 2 magnets are stuck together, the others are not.) What is the rule that explains why you labeled the poles the way you did?',
'SE_4a':'What is a heat sink?',
'EM_43b':'If Phil, a geologist, wants to test for calcite while in the field, what should he bring with him? (an acid such as vinegar). Describe what Phil should do to test for calcite and what he would observe.',
'LF_6b':'Mark an X next to each of the following that can cause more erosion in a streamtable experiment. (Use a water source with a bigger hole.) Use a water source with a smaller hole.  Make the slope flatter. (Make the slope steeper.) Explain why your choices cause more erosion.',
'II_38':'Describe what you would see if a mirror were placed along a line of symmetry on a flat object.',
'FN_17c':'Paula wants to test a food to see if it contains acid and/or sugar. When Paula added water and yeast to a sample of the food and put the mixture in a warm water bath, the mixture began to fizz and bubble. Paula tested for the other substance. What would the test results tell her?',
'FN_17a':'Paula wants to test a food to see if it contains acid and/or sugar. When Paula added water and yeast to a sample of the food and put the mixture in a warm water bath, the mixture began to fizz and bubble. What does this chemical reaction indicate?',
'ST_54b2':'The inside of a pea seed is shown below. Describe the function of each of the parts: cotyledon.',
'VB_29':'When conducting a controlled experiment, why do you use a standard?',
'MX_24':'How do you know when a solid and a liquid form a solution?',
'II_24b':'Design a way to use carbon printing to find out if 2 Labrador retrievers have the same paw patterns. How will you know if the 2 dogs have paw patterns that are the same?',
'LF_18a':'How does the water flow in a creek during a flood compared to normal water flow?',
'MX_16a':'Mariella made a saturated solution by adding a solid to 100 milliliters water. Richard made a saturated solution by adding the same solid to 250 milliliters water. Do the 2 saturated solutions have the same concentration? Explain your answer.',
'PS_45b':'Nathaniel and Sammy are trying to make up a code for sending messages to each other by dropping items in a drop chamber and listening to them. Sammy wants to design a code using blocks made of plastic, metal, and wood. Nathaniel thinks it would be a better idea to use wooden blocks of different sizes and shapes. Which student\'s method would make it easier to tell the difference between the sounds? Why?',
'MS_14b':'Look at the picture below [which shows a balance tipped down on the side containing a ball; the right cup has 20 grams and 10 grams mass pieces]. The mass of the ball is one of the following. It must be A. 20 grams. B. 30 grams. (C. 40 grams.) Explain why you chose the answer that you did.',
'ST_54b3':'The inside of a pea seed is shown below. Describe the function of each of the parts: embryo.',
'VB_1':'How do you define a controlled experiment?',
'SE_51b':'Look at the 2 solar water heaters shown here. The black triangular area is the solar collector in each container. If both heaters contain the same amount of water, would you expect the temperature change in heater X to be greater than, less than, or the same as the temperature change in heater Y after 30 minutes? (Same) Explain your answer.',
'SE_47b':'Sue bought a white wading pool for her grandchildren. She wanted to heat up the water as fast as possible. She thought that if she put plastic sheeting in the bottom of the pool it might heat up the water faster. She had a sheet of yellow plastic and a sheet of dark green plastic. What should Sue do to heat the water as fast as possible? A. Put the yellow sheeting on the bottom of the pool. (B. Put the dark green sheeting on the bottom of the pool.) C. Put no sheeting on the bottom of the pool; leave it white. Explain why you chose that answer.',
'ST_25b1':'Look at the picture of the bean seedling shortly after germination. Describe the function of each of the parts: seed coat.',
'PS_2a':'Why does a rubber band make a sound when you pluck it (pull and let go quickly)?',
'VB_5a':'Explain in one sentence what each graph tells you. Graph A) number of tree rings versus time.',
'II_26':'Look at the rubbings of 2 different fabrics. What do the rubbings tell us about the different textures of these 2 fabrics?',
'ME_38a':'Trace over the complete circuit with your pencil to show where it is. Even though there is a complete circuit, the bulb will not light. Why not?',
'MX_10':'Jorge added solid material to a container of clear colorless liquid. The mixture began to bubble. Explain why Jorge knew a chemical reaction occurred.',
'SE_10':'Bryan taped a golf tee to a globe of the Earth. He shined a flashlight on the globe. The best way for Bryan to demonstrate how shadows change during the day would be to: (rotate the globe.) move the flashlight. Why would this best demonstrate how shadows change during the day?',
'MS_39':'Why is it important to use a standard unit when making a measurement?',
'II_13b':'Look at the finger patterns below, then answer the questions. What is different about the finger patterns?',
'PS_26p':'Kate said: "An object has to move to produce sound." Do you agree with her? Why or why not?',
'LF_27a':'The change in elevation between contour line A and contour line B is 20 meters. What is the change in elevation between contour line A and contour line C? How do you know that is the change in elevation between A and C?',
'ST_25b2':'Look at the picture of the bean seedling shortly after germination. Describe the function of each of the parts: cotyledon.',
'VB_5b':'Explain in one sentence what each graph tells you. Graph B) speed of light versus time.',
'EM_45c':'You used several methods to separate and identify the substances in mock rocks. How did you know the crystals were salt?',
'MX_1':'Gerry used a paper filter to separate a mixture of sand, salt, and water. Think about particle size and answer this question: Why did the salt go through the filter while the sand got caught?',
'LF_34b':'Draw an arrow to show which direction the water flows in the river. (Rose Mountain is on the left and Happy Valley is on the right in the drawing.) Why do you think the water flows in that direction?',
'EM_45b':'You used several methods to separate and identify the substances in mock rocks. How did you separate the salt from the water?',
'EM_27b':'You can tell if a rock contains calcite by putting it into a cold acid (like vinegar). Describe what you would observe if you did the acid test on a rock that contains this substance.',
'FN_20b':'Anna wants to compare the fat in walnuts and peanuts. Using only the results from her experiment, how will Anna know which one, walnuts or peanuts, has more fat? The experiment has the following procedure: Measure out equal masses of walnuts and peanuts (for example, one gram). Smash the samples uniformly in the same size area on brown paper. Wait (the same time for each).',
'MX_53':'Adam\'s friend gave him a cup filled with a water solution. Adam did not know what solid material was used to make it. He evaporated the solution and found crystals in the dish after all of the water was gone. How will the crystals help him decide what solid material was used to make the solution?',
'ME_73':'Wendy is making an electromagnet. She wrapped a long, insulated wire around an iron nail. What should Wendy do next to complete the electromagnet?',
'LF_39':'Don wants to make a map of his neighborhood that fits into his notebook. He has a photograph of the neighborhood but it is too large. Complete the description of how Don can draw an accurate map that fits into his notebook: 1. Put an overlay grid over the model. 2. ____ 3. Draw a smaller grid with the same number of squares in your notebook. 4. ____ 5. Create a key to go with the map.',
'VB_12d':'Describe the relationship between the length of the string and the number of swings a swinger makes in 15 seconds.',
'FN_27b':'Jana and Emily used indophenol to test 3 beverages for vitamin C. The number of drops they added to make a 10 milliliters sample of the blue indophenol indicator become colorless are given here: 8 drops of Overly Orange 13 drops of Luscious Lemon 33 drops of Fruitsnap (and it still didn\'t change color). Why didn\'t the indophenol change color with Fruitsnap?',
'FN_20a':'Anna wants to compare the fat in walnuts and peanuts, but she isn\'t sure how. Write a procedure Anna can use to compare the fat in walnuts and peanuts.',
'ST_52a':'When a seed germinates, why does the root grow first?',
'SE_25a':'The graph shows temperature data from 2 containers. One container had 100 milliliters of water and the other had 100 milliliters of dry soil. Each container was placed in the sun for 20 minutes and then in the shade for 20 minutes. Which container, A or B, had the dry soil? Explain how the graph helped you decide which container had the dry soil.',
'LF_33b':'Below is a drawing of a river that carried sand, clay, and pebbles (past the river\'s mouth) to a lake. Why is the clay deposited where you indicated?',
'MX_47b':'Tabby made 2 solutions using the same solute and water. [Cup one has 6 spoons solute in 50 milliliters water. Cup 2 has 8 spoons solute in 50 milliliters water. Both solutions are clear with material on the bottom.] Which sentence below describes the concentrations of the 2 solutions? (A. The solutions in the cups have the same concentration.) B. Cup one has the more concentrated solution. C. Cup 2 has the more concentrated solution. D. There is not enough information to answer the question. Explain your choice.',
'SE_48':'Maggie wanted to find out if surface area affected temperature change. She had 3 white cake pans. She filled each with 300 milliliters of water. She put one small black plastic disk in pan A, 2 in pan B, and 3 in pan C. Then she put all 3 pans in the sun. When Maggie measured the water temperature in each pan after 15 minutes, which pan do you think held the hottest water? Explain your answer.',
'FN_27a':'Jana and Emily used indophenol to test 3 beverages for vitamin C. The number of drops they added to make a 10 milliliter sample of the blue indophenol indicator become colorless are given here: 8 drops of Overly Orange 13 drops of Luscious Lemon 33 drops of Fruitsnap (and it still didn\'t change color). Does Overly Orange or Luscious Lemon contain a higher concentration of vitamin C? What is the evidence?',
'MX_46b':'Nigel made solutions in the bottles as shown below. Solution A was 2 spoons solute in 50 milliliters water and was clear with no material on the bottom. Solution B was 4 spoons solute in 100 milliliters water and was clear with no material on the bottom. Nigel took 50 milliliters of solution from bottle A and put it in cup A and 50 milliliters of solution from bottle B and put it into cup B. He put the cups on each side of the balance. Which picture shows what would happen? A is heavier than B, B is heavier than A, (A and B are the same weight). Explain your choice.',
'PS_4ap':'Darla tied one end of a string around a doorknob and held the other end in her hand. When she plucked the string (pulled and let go quickly) she heard a sound. How would the pitch change if Darla pulled the string tighter?',
'EM_48b':'Emily evaporated a solution and drew a picture of the crystals that remained. Using the crystal-shape key below, what was dissolved in the solution? A. Epsom salt (B. alum) C. sodium thiosulfate.  Explain why you chose that answer.',
'VB_42':'Don wanted to find out if a battery powered toy truck would always travel the same distance in 10 seconds no matter how heavy a load the truck carried. He placed one or more blocks (the load) in the back of the truck and recorded his results. (Each block had the same mass.) The following graph displays his results. What is the relationship between the number of blocks and the distance traveled by the truck in 10 seconds?',
'ME_79':'Kurt was investigating which objects stick to magnets. He made an entry in his science notebook and drew a picture to help explain what he did. Look what I did! I picked up a paperclip with a magnet. Then that paperclip picked up another one, and then another one. And they weren\'t hooked together either. All they had to do was touch each other. Explain to Kurt why he was able to pick up all those paperclips, even though the magnet was only touching the first one.',
'II_13a':'Look at the finger patterns below, then answer the questions. What is the same about the finger patterns?',
'LF_26a2':'Pam and her family were planning a hike. Pam found 2 maps of the same area. Her friend recommended she use the topographic map when they went to the lake. Why would Pam\'s friend recommend using the topographic map for the hike?',
'PS_4bp':'Darla tied one end of a string around a doorknob and held the other end in her hand. When she plucked the string (pulled and let go quickly) she heard a sound. How would the pitch change if Darla made the string longer?'
}

q_rubric_dict = {
'ST_59':['Elena should include a separate shelter for each lizard.'],
'MX_41':['Yes, they could both be correct. Solubility is not the same for all materials.'],
'FN_24c':['Diva needs to know how much gas pure sugar produced in order to decide whether the cookies are pure sugar. Diva needs to make a controlled experiment.'],
'SE_24a':['In the afternoon, shadows point east because the Sun is in the west. So the west side is opposite of the shadows.'],
'FN_24b':['No, Fruity cream cookies are not pure sugar. The cookies did not produce as much gas as was produced by the pure sugar.'],
'MX_36b':['A reaction was indicated by the bubbles of gas (a new material) being produced.'],
'SE_31b':["A. The water in Julie's heater got hotter faster (or had a greater temperature change in 20 minutes). The collector in heater A has the greater surface area so the water in A would get hotter faster than the water in B."],
'LF_26b2':['The contour lines are closest together at Q.'],
'VB_40a':['The boat with 0 marbles is the standard in this experiment.'],
'MS_64a':['Greta and Andre have different sized hands.'],
'EM_46':['The crystals have different shapes. Salt crystals are square with little Xs. The calcite crystals are shaped like needles (long and skinny).'],
'ST_58':['If a part of the plant contains seeds, that part is the fruit.'],
'LF_31b':['Earth materials erode and are carried by flowing water (a river). The earth materials form a delta when the materials are dropped off or deposited when the water stops flowing (at the mouth of a river).'],
'MX_36a':['A solution was created when the eggshells disappeared (dissolved) and the mixture ended up as a clear liquid.'],
'PS_15bp':['The saxophone has the longest and fattest tube.'],
'MS_30b':['Use a standard unit of measurement such as centimeters or meters.'],
'ME_78b':['The motor will continue to run because the motor still has a pathway to the D-cell.'],
'PS_51a':['When the string was tighter, the pitch was higher.'],
'SE_22b':['Solar water heaters with blue collectors heat up faster than solar water heaters with red collectors.'],
'MS_43b':['Use a standard unit of mass like grams.'],
'SE_45':['In the morning the Sun is in the east so shadows point west. In the afternoon the Sun is in the west so shadows point east.'],
'MX_49':['X. When equal amounts of 2 solutions are compared, the solution that is more concentrated will have the greater mass (weigh more). 50 milliliters of X has a greater mass than 50 milliliters of Y, so X must be more concentrated.'],
'ME_66a':['No, string is an insulator, so the pathway would not be completed.'],
'MX_22a':['A solution is a mixture formed when a solid dissolves in a liquid.'],
'SE_22c':['When 2 variables are changed, it is impossible to determine which variable is causing what effect.'],
'PS_51b':['When the string was shorter, the pitch was higher.'],
'SE_22a':['Water heats up faster when covered.'],
'ME_5b':['The brass ring would not stick to the nail because the ring is not iron. The nail is a temporary magnet, but brass cannot become a temporary magnet and thus does not stick to magnets.'],
'EM_47':['The sand particles are larger and settle first. The flour particles are smaller and therefore settle more slowly.'],
'MS_43a':['The marbles used by the 2 groups were probably different masses.'],
'SE_46':['Yes. Black absorbs more heat (energy) than white. White reflects more heat (energy) than black. Therefore the black covered box will heat up faster than the white covered box.'],
'II_20b':['Seeing multiple colors would show that vinegar can separate the permanent ink.'],
'ME_66b':['Yes, aluminum foil is a metal and all metals conduct. The aluminum foil would complete the circuit.'],
'MS_50a':['Bill must have been using degrees Celsius if 28 is a warm temperature.'],
'ME_7a':['Lee should put one of the loose wires on one part of the object and the other loose wire on another part of the object and see if the object completes the circuit.'],
'SE_16b2':['20 centimeters size is between 10 and 30 centimeters, so the temperature change must be less than the 30 centimeters container, but more than the 10 centimeters container. 6 degrees Celsius is halfway between.'],
'MX_11e':['Set up evaporation dishes. The water will go into the air (evaporate) and the solid will be left in the dish (crystallize).'],
'EM_35':['The harder coin will scratch the other.'],
'LP_16d':['You have to pull up in this system.  In the single fixed pulley system, you get a directional advantage.'],
'ME_17b':['There is a complete circuit connecting the bulb to the D-cell battery.'],
'EM_21a':['Rub the minerals together and see which one scratches the other.'],
'MX_52b':["Emily's crystals are the same shape as the alum crystals shown on the key."],
'ME_17c':['The motor will run.'],
'PS_44':['If the string is shorter, the pitch will be higher. If the string is longer, the pitch will be lower.'],
'SE_3c':['The tree blocks the light from the Sun.'],
'PS_12':['The water splashed because the fork was vibrating. Vibrations make sounds.'],
'ME_7b':['If the motor runs, the object is a conductor.'],
'MX_11f':['Jim would see a clear solution. 3 spoons of solid dissolves, so 2 spoons will dissolve.'],
'ME_17a':['The bulb will light.'],
'EM_21b':['The harder mineral will leave a scratch on the less hard mineral. If the black mineral is harder, the brown mineral will have a scratch.'],
'VB_22c':['When less than 3 scoops of fertilizer are used, the more fertilizer used, the greater the mass of beans harvested. When more than 3 scoops of fertilizer are used, the more fertilizer used, the less the mass of beans harvested.'],
'MX_42a':['The particles of the solid are too big to go through the holes in the paper filter.'],
'ME_30':['A car is made of a lot of iron or steel.'],
'LF_13a':['Earth materials settle out during deposition.'],
'ME_65a':['If the middle bulb burns out the other 2 bulbs will stay lit.'],
'ME_69b':['You can not tell because we do not know the strength of the magnets. If the magnets are weak, the second piece of foam might put too much distance between the 2 magnets for the interaction to hold the magnets in place. If the magnets are strong the magnets might stay in place.'],
'LP_15c':['If the effort is applied farther from the fulcrum, less effort will lift the load.'],
'MX_11c':['The concentrations are the same because both mixtures are saturated.'],
'ME_17d':['There is a complete circuit connecting the motor to the D-cell battery.'],
'MX_18':['A chemical reaction occurred. [The white solid is a precipitate (a new material) which indicates the reaction.]'],
'ME_10':['The negative terminals of the D-cell batteries are facing each other, so there is no flow of electricity.'],
'VB_15c':['All variables except the girl need to be held constant (controlled). The girls all need to throw the same ball and use the same launch technique.'],
'VB_15b':['Different girls are throwing the balls. The girls use different launch techniques. Each girl uses a different type of ball.'],
'MX_19':['The fizzing indicates a gas, a new material, is produced. When a new material is produced, a chemical reaction has occurred.'],
'ME_17e':['No. Only one circuit can be complete at a time.'],
'ME_27b':['If the key sticks, the key is iron; if the key does not stick the key is aluminum.'],
'EM_26':['The salt crystals are squares with Xs on top.'],
'FN_19b':['Susan should compare the amount of gas in each bag. The bag with the most gas contains the food with the most sugar.'],
'ME_6b':['The circuit is not complete. The electrical pathway is broken.'],
'ME_72':['That circuit is the only circuit that uses all of the correct contact points.'],
'EM_16b':['Topaz would be the hardest of the 4 minerals because none of the tools could scratch topaz and at least one tool could scratch each of the other minerals.'],
'ME_65b':['The circuit is parallel, so each bulb has its own pathway to the D-cell battery, so the bulbs that do not burn out continue to shine.'],
'VB_15a':['The launch position (line on the playground) is the variable that is controlled.'],
'MX_11a':['The mixtures are all clear.'],
'LF_28a2':['You can see how much time is taken for the earth materials to move.'],
'PS_46b':['So that people notice the sound.'],
'ST_31b':['The elytra are used to make sounds.'],
'ME_28b':['Like poles repel and opposite poles attract.'],
'SE_4a':['A heat sink is any material that absorbs (a lot of) heat.'],
'EM_43b':['Put acid on a rock. If the acid fizzes, Phil would know that the rock has calcite.'],
'LF_6b':['A bigger hole puts more water in the stream. Increased flow moves more earth materials, increasing erosion. A steeper slope makes the water flow faster. Faster flow moves more earth materials, increasing erosion.'],
'II_38':['The mirror would complete the image of the object and make the object appear whole.'],
'FN_17c':['If gas (carbon dioxide) is produced, the food contains acid. If no gas is produced, the food does not contain acid.'],
'FN_17a':['The food contains sugar.'],
'ST_54b2':['The cotyledon provides food for the plant to begin growing.'],
'VB_29':['A standard is used for comparison to determine how changing one variable changes the results.'],
'MX_24':['The solid dissolves (seems to disappear) and the liquid remains clear.'],
'II_24b':['Compare prints to see if the patterns are the same.'],
'LF_18a':['During a flood there is more water in the creek and the water flows faster.'],
'MX_16a':['Yes. If both solutions are saturated (and at the same temperature), the solutions will both have the same concentration. Once a solution is saturated, the solution cannot get any more concentrated.'],
'PS_45b':["Sammy's. Different materials would make very different sounds. Different size wood would make very similar sounds."],
'MS_14b':['There is 30 grams (20 grams plus 10 grams) in the right cup. The balance is tipped down on the ball side which means the ball weighs more than 30 grams. 40 grams is the only weight heavier than 30 grams.'],
'ST_54b3':['The embryo contains the tiny leaves and root of the plant.'],
'VB_1':['An experiment is controlled if only one variable is changed at a time.'],
'SE_51b':['The black areas in the 2 collectors (and the collectors themselves) are the same size. The amount of heat absorbed will be the same so the temperature change will also be the same.'],
'SE_47b':['Darker colors absorb more heat than lighter colors. The dark green sheeting is the darkest color available so she should use the green sheeting.'],
'ST_25b1':['The seed coat protects the seed until the plant begins to grow.'],
'PS_2a':['The rubber band vibrates.'],
'VB_5a':['As time increases, the number of tree rings also increases.'],
'II_26':['Fabric A is smoother and has a finer texture. Fabric B is rougher and has a coarser texture.'],
'ME_38a':['The circuit does not include both contact points on the bulb holder, so the electricity can not go through the bulb.'],
'MX_10':['Bubbling means a gas has formed. Gas is a new material. A new material indicates a reaction.'],
'SE_10':['Shadows change during the day because the Earth is turning. The Sun stays in one place.'],
'MS_39':['So people know exactly what the measurement means. A standard is necessary so everyone measures using the same unit and everyone gets the same measurement.'],
'II_13b':['The loops begin at opposite sides of the print.'],
'PS_26p':['Agree. Vibrations are movements. Vibrations produce sound.'],
'LF_27a':['40 meters. The change in elevation is 20 meters between A and B and 20 meters between B and C. So the change between A and C is 40 meters. 20 m plus 20 m equals 40 m.'],
'ST_25b2':['The cotyledon provides food for the plant until the plant can make food for itself.'],
'VB_5b':['As time increases, the speed of light stays the same (remains constant).'],
'EM_45c':['The crystals were square with Xs on the surface.'],
'MX_1':['The dissolved salt particles are small enough to go through the holes in the filter paper, but the sand particles are too large.'],
'LF_34b':['Water will always flow downhill (mountain to valley).'],
'EM_45b':['The water was evaporated, leaving the salt.'],
'EM_27b':['Many tiny bubbles will rise from the calcite when the calcite contacts the cold acid.'],
'FN_20b':['Anna should compare the size of the grease spots. The sample with the larger grease spot has more fat.'],
'MX_53':['Crystals of different materials have different shapes. So the shape will help identify the material.'],
'ME_73':['Wendy should hook one end of the wire to the negative side of a D-cell battery and the other end to the positive side of the D-cell battery.'],
'LF_39':['2: Trace the features from the photograph onto the overlay grid. 4: Transfer the features to the matching squares on the smaller grid in your notebook.'],
'VB_12d':['The shorter the string is, the more the string swings in 15 seconds.  The longer the string is, the less the string swings in 15 seconds.'],
'FN_27b':['Fruitsnap contains little or no vitamin C so Fruitsnap does not change the color of the indophenol.'],
'FN_20a':['Measure out equal masses of walnuts and peanuts. Smash the samples uniformly in the same size area on brown paper. Wait the same time for each sample.'],
'ST_52a':['The root grows first so the root can take up water for the plant.'],
'SE_25a':['A. Dry soil heats more quickly and cools off more quickly than water. The graph shows A heats and cools more quickly than B, so A must be the dry soil.'],
'LF_33b':['The clay particles are lighter or smaller and are therefore carried farther by the water, so the clay particles end up the greatest distance away from the mouth of the river.'],
'MX_47b':['Both solutions are saturated (as indicated by the material on the bottom) so, even though one cup has more spoonfuls of sugar, both have the same (maximum) concentration.'],
'SE_48':['C. Black absorbs more heat (energy) than white. Pan C has the most dark surface area so C would heat up the fastest and have the highest temperature.'],
'FN_27a':['Overly Orange takes fewer drops to change the color of indophenol so Overly Orange has a higher concentration of vitamin C.'],
'MX_46b':['The solutions have the same concentration (2 spoons in 50 milliliters is the same as 4 spoons in 100 milliliters) so the mass would also be the same.'],
'PS_4ap':['When the string is tighter, the pitch will be higher.'],
'EM_48b':['The crystals are the same shape as the alum crystals.'],
'VB_42':['The more blocks the truck carries, the less distance the truck travels in 10 seconds.'],
'ME_79':['When a paperclip sticks to a magnet, the paperclip turns into a temporary magnet, so the paperclip can stick to other paperclips and other things made of iron.'],
'II_13a':['Both finger patterns are loops.'],
'LF_26a2':['She recommended the topographic map because the map shows the elevations along the trail. Pam would know where the trail was the steepest.'],
'PS_4bp':['When the string is longer, the pitch will be lower.']
}